End of W4

Time is flying by quickly but being in RC has been a good forcing function for me to just start building and then to keep going. In the past, I would have taken a lot longer to get started especially when the project idea felt hard to grasp. Breaking things down into smaller parts and working on those parts has been helpful. Also, it feels like as I work, I increasingly have no shortage of ideas of things to build and things I want to learn which makes it tough to wrangle it all in the limited time here.

This past week I had a nice chat with @Rachel Vincent Petacat (she) (Faculty), paired program with @Grace Harper (she) (SP1'26) on training a neural network, and learned about red teaming with @John Aiken (he) (SP1'26). He is also interested in world modeling for science so we are going to brainstorm more about it. Very exciting! I also learned about lean programming language for verifying mathematical proofs from @Patrick Hulin (he) (SP1'26) and @Eva Khoury (they) (Faculty) geo-guesser game she is building for NGW! On Friday, I played Zork with the Zork crew. The highlight was when we died and became ghosts in the game and had to pray at the alter to get our bodies back.

During the hour of hard things on Tuesday, I read up on training distributed models. And in the process, also looked into why Spark (another distributed framework) is not regularly used in data distributed training. From what I understand, it is because Spark is more CPU-centric and DDP was built to be more GPU-centric. For instance, Spark uses JVM while DDP uses C++/cuda so there is no serialization of data when using DDP making the transfer process much quicker which Spark would require when going from CPU to GPU during the training process.

I gave my first RC presentation on Thursday about distributed training which was fun because putting together a presentation crystallized some concepts for me as well in the process when thinking about how to communicate what I have done and learned.

I also tried drawing a husky using P5.js for the rapid riter. That was HARD and my dog looks ugly lol. But it was a fun hour just messing around with something new and different from what I am used to.