I thought it might be useful to compare some output from the full uncompressed version of ds4f (on cline-pass in this case):
https://com-pute.com/nick/rubiks_cube.html
and exported Pi session in which that app was built:
To a version of the same app, created with the 2 bit compressed version that runs in the ds4 engine, on a single DGX Spark (Asus GX10):
https://com-pute.com/nick/rubiks--dsf4-local.html
and the Pi session in which it was created:
I love that this tiny box which sits on the floor (and could fit in a handbag), can reliably write working code like this. And I love that I'm able to use the same model locally that I use on the cline-pass API. I'm also fully aware of how good a buy that cline-pass API is. I use the API version all day every day, on an account that costs less than $7 per month.
You can see the difference in quality between the quantized local version and the uncompressed version running on cline-pass. If you take a brief look at the session which used the locally hosted quantized version, the model required many more iterations, needed guidance completing the task, and did not create as nice of a final application as the uncompressed version on the API (more features were added to the app created by the uncompressed LLM, and the UI looked better in that app).
Quantized versions of models are like drunk versions of themselves. They have the same background as their full precision versions, but they make more mistakes in judgement and have trouble thinking things through as deeply.