Amit Israeli

I train generative models, fit them onto small devices, and build toys out of them.

AI Research Scientist at Wix, working on diffusion, multimodal models, and the engineering it takes to get an idea from paper to production. Previously at Reality Defender, NLPearl and Pashoot Robotics.

See the latest research Download CV Get in touch
  • TrainingES-EGGROLL, text-to-image post-training with evolution strategies
  • PlayingMaple LLM, a 230M model that plays MapleStory from pixels
  • BuildingKoala Controls, a macOS suite for my desk and couch
Cartoon of Amit holding a golden retriever, standing with the cast of Silicon Valley

A tiny VAE is really training in your browser. Click a digit to resample it.

Amit Israeli, drawn Ghibli-style with a dog

I work across computer vision and NLP, focused on multimodal models and text-to-image / text-to-video diffusion. My work sits between research and engineering: building data and training pipelines, running rigorous evaluations, and getting models to run on a single GPU and, when needed, on the edge.

Email LinkedIn GitHub Hugging Face

Experience

From robotics to deepfake detection to generative models, 2023 to now.

  1. Mar 2026 – Present

    AI Research ScientistCurrent

    Wix

    Making Websites beatiful — not like this one ):

    Generative AI Multimodal Research
  2. Jan 2025 – Jan 2026

    Computer Vision Research Engineer

    Reality Defender

    Developed and optimized deep-learning computer-vision systems for deepfake and synthetic-media detection across image, video, and video+audio. Multi-model architectures combining specialist detectors with shared backbones for production-scale inference.

    Deepfake detection Multimodal Production CV
  3. Dec 2024 – Apr 2025

    Computer Vision Research Engineer

    LuckyLab · Freelance

    Edge-optimized segmentation and detection for production in a few-shot domain. Converted research models to production graphs (JAX, TensorRT) with quality gates using pruning and quantization.

    Edge deployment Segmentation Few-shot JAX / TensorRT
  4. Jun 2024 – Jan 2025Tel Aviv

    Deep Learning Research Engineer

    NLPearl

    Compact language models for multi-task outputs, audio tokenizers and LLMs for audio tasks, and real-time conversational pause detection with fine-tuned LLMs. LoRA and multi-stage training under tight latency budgets.

    LLMs Real-time LoRA Audio
  5. May 2023 – Jun 2024Rehovot

    Computer Vision & Deep Learning Research Engineer

    Pashoot Robotics

    Zero/few-shot improvements with SAM, YOLO-World, Grounding DINO, and CLIP. Multi-object tracking and 6-DoF pose estimation using synthetic data and domain randomization, plus 3D reconstruction (NeRF, Gaussian Splatting, image-to-3D) for simulation.

    Zero / few-shot SAM / YOLO-World 6-DoF Sim / Domain rand. Robotics

Toys that learn

Hands-on ML: drive an optimizer, out-judge a generator, compose Funko Pops with my VAR model, and chat with a tiny LLM that lives in your browser. Rumor has it not everything here is listed.

guide the optimizer to the global minimum

Descent: you are the optimizer

SGD with momentum, played by hand. Pick a learning rate, roll the ball to the flagged global minimum. Local minima are traps, and too much lr ends in NaN.

Generated Funko preview
A Funko Pop figure of styled as a performing .

rumor: one combination is legendary ⚔️

Funko Forge: PopYou2, live

Real outputs from my VAR text-to-image model. Compose the prompt, meet the figure. 36 combinations, zero GPUs harmed.

real ML papers vs. titles a model made up. Can you tell?

Real or Generated?

The deepfake-detection problem, reduced to its purest form: ML paper titles. Half are real published research. Half were hallucinated by a language model for this site.

the weights download once, cache in your browser, and every token is generated on your device

WebGPU preferred, WASM fallback. Answers ≠ facts.

Tiny Amit: a 230M-parameter me

LiquidAI LFM2.5-230M, quantized and running entirely in your browser. No server, no API. It has read a short note about me, so ask it anything. At 230M parameters it is fast, charming, and confidently wrong on schedule.

Projects

Research and engineering work. Filter by domain.

SANA-Video Tom & Jerry LoRA sample

Tom & Jerry: SANA-Video LoRA

Fine-tuning SANA-Video 2B with LoRA to generate Tom & Jerry-style animations at 224×224 — from class-only LoRA to scene-aware text conditioning, all on a single consumer GPU.

T2V LoRA VLM + vLLM Diffusion video
Read the details

Versioned training roadmap

  • V1 — Class-only LoRA @ 224×224. Freeze SANA-Video 2B and train LoRA on all linear layers using a single class prompt over ~16k cached 5s clips (81-frame latents) with flow-matching.
  • V2 — Scene-aware LoRA with VLM labels. Use Qwen3-VL via vLLM to generate structured scene descriptions (ENVIRONMENT / CHARACTERS / PROPS / ACTION / CAMERA), then condition SANA-Video on per-clip text prompts.

V1 checkpoint progression

Same seed and class prompt; only the LoRA checkpoint changes.

Tom & Jerry checkpoint preview

Base model: off-style 224×224 sample before LoRA fine-tuning.

Future directions

Next: distillation, pruning, and quantization for deployment, plus more systematic evaluation (identity consistency, motion quality) and RL-style post-training to better align generations with scene descriptions.

Research log GitHub Hugging Face W&B report
SigLIP–zoom method overview

Sana Simplified: Image & Zoom Control

Research playground around Sana 1.5 and Sana Sprint: ControlNet implementation and a SigLIP-driven zoom controller that smoothly changes camera distance while keeping identity and style consistent.

T2I Diffusion ControlNet 3D rendering
Read the details

Method overview

A zero-zoom reference image is encoded with SigLIP into an object token, while a scalar zoom value z ∈ [0, 1] is mapped through a small MLP to a zoom token. Both tokens are appended to the cached text encoding, and the Sana 1.5 transformer is fine-tuned with LoRA.

Reference & zoom sweep

Reference zero-zoom
Reference zero-zoom render (zoom_0)
Zoom sweep
Generated zoom sweep as zoom token moves 0 → 1
GitHub Hugging Face
Kokoro coefficient trajectory

Kokoro: Tiny TTS, Big Voices

Small-footprint TTS with Kokoro-82M: fast mixture-of-voices coefficients, full-embedding optimization, and visual tooling (trajectory visualizer + export).

TTS Embedding opt Visualizer
Read the details

Highlights

  • Mixture-of-voices coefficient optimizer with temperature-annealed softmax.
  • Optimizing voice embedding and LoRA in few-shot scenarios.
  • GUI visualizer: circular mixer + per-voice bars + MP4 export.
GitHub W&B report
Token Budget example

Token-Budget-Aware Reasoning for VLMs

Built on Token-Budget-Aware LLM Reasoning, extended to multimodal setups by combining a frozen SigLIP image encoder with a LoRA-tuned LLM that predicts a reasoning budget prior to decoding.

CoT Multimodal Paper impl.
Read the details

Summary

The original paper asks whether we can control chain-of-thought with an adaptive token budget. This variant predicts the budget for multimodal inputs and constrains decoding accordingly.

Deep dive

  • Architecture: frozen SigLIP + budget head fused with LLM states.
  • Training: LoRA fine-tuning with oracle CoT lengths; KL regularization.
  • Evaluation: accuracy vs. average tokens; ablations on freezing / head depth.
Token budget predictor diagram
GitHub
VAR Explained

PopYou2: VAR Text-to-Image

Adapted Visual AutoRegressive (VAR) for Funko Pop! generation with a custom "doll" embedding and a SigLIP→text adapter, enabling I→I and T→I paths.

VAR T2I Autoregressive
GitHub Hugging Face Space Try the live builder
SAM LoRA figure

Few-Shot SAM + LoRA

Adapted SAM and efficient variants (EdgeSAM, TinySAM) with LoRA for class-aware few-shot segmentation; improvements over PerSAM and edge deployment.

Few-shot LoRA Pruning Quantization Edge
Read the details

Deep dive

LoRA-adapted SAM variants with compression for edge deployment, with experimentation in extreme few-shot scenarios.

COCO Cityscapes Soccer SAM variants
GitHub

CelebrityLook: Mobile Face Transform

On-device face stylization with a distilled StyleGAN2 and CLIP-to-StyleGAN latent alignment; ~30 fps on modern phones.

Edge GAN CoreML Latent alignment
GitHub
FastGAN sample

PopYou: FastGAN + CLIP

Multi-stage pipeline with FastGAN and CLIP inversion for promptable Funko-style synthesis.

GAN CLIP inversion Synthetic data
GitHub
MusicGen visualization

MusicGen: Genre LoRA

LoRA-adapts MusicGen for genre-specific generation while keeping prompt controllability.

LoRA Music generation
GitHub

KoalaReadingAI: Papers as Podcast

Pipeline that turns AI research papers into short audio episodes published to Spotify and YouTube.

TTS Pipeline
Listen on Spotify YouTube channel

Koala tools

Native apps I build for my own desk and couch. They share one action engine and one very helpful koala.

Koala Controls

Your desk. Your couch. One very helpful koala.

amit154154/koala-controls Release v1.20.1 Install guide

Press any key. For corners, tap two keys together: 10+11 top left, 12+11 top right, 10+9 and 12+9 for the bottom.

Your desk, simulated. Each key runs the same action it runs on my Mac.

Three ways in, same actions underneath

Both Mac apps share one signed host, one action engine and one phone pairing system. Here is the real thing.

The illustrated Command + Option shortcut guide on a MacBook

At the deskMini Keyboard

Map a USB macro pad with per-device remapping, no driver. On a bare MacBook, hold Command + Option.

Explore the keyboard
Koala TV home screen with streaming services and a PS3 shelf

On the couchKoala TV

A TV-friendly home for YouTube Music, Spotify, Netflix, NBA and more. It resumes what is already playing instead of reloading it.

Explore the TV
Koala Remote on an iPhone: TV controls and a D-pad

In your pocketKoala Remote

An old iPhone becomes a shortcut pad, trackpad and TV remote.

Pair your phone
runs onmacOS 13+, iPhone pairing on 15+ built withSwift, AppKit, SwiftUI, IOHID, Network.framework TLS phonevanilla JS PWA over local HTTPS licenseMIT

Recent papers

Synced from my Zotero library. Papers I'm currently working through.

    Get in touch

    Open to research collaborations, side projects, and interesting problems.

    amit1541541@gmail.com
    LinkedIn GitHub Hugging Face