2020-09-20

@LorenzoTeixeir4 @Zog_Fot_Pik @samiralyateem @ID_AA_Carmack True

@IgorAntarov @soumiksf @ID_AA_Carmack Custom ultra high speed bus & efficient cooling

@teslaownersSV @jgrano305 @soumiksf @ID_AA_Carmack Ok

@flcnhvy @soumiksf @ID_AA_Carmack Pretty much any machine-learning

@jgrano305 @soumiksf @ID_AA_Carmack It’s big

@Zog_Fot_Pik @samiralyateem @ID_AA_Carmack Sure

@soumiksf @ID_AA_Carmack Yeah, we will open Dojo for training as a web service once we work out the bugs

@samiralyateem @ID_AA_Carmack There are a lot of pointless zeroes in FP32 neural nets. You can chop off 16 of the 32 bits without losing meaningful precision. This works well with neural nets, but not with regular computing, which expects extreme precision.

@soumiksf @ID_AA_Carmack Dojo uses our own chips & a computer architecture optimized for neural net training, not a GPU cluster. Could be wrong, but I think it will be best in world.

@llamaa35079089 @ID_AA_Carmack @Tesla You really want waypoints this much?

@ID_AA_Carmack For Dojo, we’re betting on FP16 with floating floating point (essentially, truncating mostly zeroes), so it feels like FP32 & existing software just works. We considered FP8 for a while, but everyone hated it. I even asked Twitter! For inference, int8 works great.

@WholeMarsBlog Yes

@BabyTesla3 @WholeMarsBlog Yes

@WholeMarsBlog There’ll be lots of green space around factory, but building itself is continuous. The “open” areas inside are covered. They’re internal semi truck roads inside a giant monolithic building.

@SavedTesla @peterdog15 Shaped like a diamond & aligned on true north