End of W5
Building (mostly vibe coding) "Where is my treat" arcade game and then seeing it actually play on the rcade cabinet was really cool. And learning how the rcade all came together during localhost from the people involved was even cooler.
This was my first time building something with a GUI and learning about sprites and rendering concepts like flip and blit.
And it also reinforced my understanding of asyncio, await, and async which was also a common concept when building agents with MCP but generally not common when working with core ML training loops where everything is synchronous.
I thought it was interesting that LLMs struggle with sprite animation. From my video, you can see that the dog's tail was going from left -> middle -> right on repeat instead of left -> middle -> right -> middle -> left. At first, I thought the LLM had mixed up the order of the sprite sheets. But on closer inspection it was two errors coupled together.
It had ordered the sprite list as tail middle, tail left, tail right and then repeat from the start of the list with equal timing between sprites. So the tail left to tail right transition looked much quicker (almost non existent) than the tail middle to tail left transition
Except I had mislabeled the sprite frames and tail left should be tail right and vice versa. And the LLM did not know better to match the contents of the sprite with the title of the sprite.
The fixes should now be reflected on the current rcade cabinet console as well as on rcade.dev.
Walked through some fine-tuning concepts and projects with @Tawfiq Hamid (he) (SP1'26) @c stavridis (they) (W2'26) @Parker Wilf (he) (SP1'26). They came with lots of questions which was nice and interactive and I hope it inspired them to try fine-tuning something.
Nice chat with @Isha Bhand (she/they) (W2'26) where I got to interact more with her LLM-based Zork game and learn about ML at Youtube Music. I also participated in the weekly Zork adventure game where we made some progress in the cyclops room and also found gunk which seems like it could be important.
Paired with @John Aiken (he) (SP1'26). He showed me an interesting time series dataset of chaotic systems and we visualized the data to see if we could model it with JEPA. Also thank you for explaining all these science concepts to me!
Depending on initial conditions (IC) , the trajectory of the same system looks very different due to varying boundary conditions.
For instance, with IC = [-9.7869288 -15.03852 20.533978] we get the following time series..
But with IC = [-9.7869288 -15.03852 20] we get this other time series..
[continuing]
And initially, we formulated the problem to be given 500 different initial conditions and trajectories, predict attractor wells in phase space for each initial condition. But when we used dynamical systems library in Julia to get attractor wells coordinates, we realized that the system had the same attractor well coordinates regardless of initial conditions. So...we are going to have to reformulate our problem.
Also seems like this system can be solved analytically...and is deterministic..so I am not sure JEPA is the right tool for the task. In school, when I took probabilistic time series, I learned that sometimes classical methods like ARIMA or GP go a long way for time series modeling and applying deep learning sometimes improves the forecast but its mostly over-engineering (at least at that time -- Dec 2024 -- and where AI was at).
Still working on distributed model training. More details later..
