replace broken links (fixes #10)
Build & Push Hugo Site Image / Build & Push Image (push) Successful in 13s

This commit is contained in:
2025-08-02 19:56:14 -05:00
parent 2865ed351f
commit acc732ba4d
@@ -15,8 +15,11 @@ I impulsively purchased a NVIDIA RTX 4060 at my local Best Buy. I thought it wou
LLMs continue to improve apace, getting smaller, more efficient, smarter, generally better in every way, every day, relentlessly and with such speed that I really cannot even pretend to have kept up with their developments to any meaningful degree. I merely absorb the general *sense of things*, passively, allowing the vibes of the techno-accelerationist-sphere wash over me. Every week a new set of capabilities are launched with a new version of a new model using a new paradigm. Exciting, truly what a time to be alive. Around the time of this writing, Deepseek released its R1 model which one can run on relatively low-spec hardware (this is true of many earlier models by other organizations, but none produced the same frenzied excitement as this one), so I hopped on the hype wagon and now I have a somewhat powerful reasoning model running on my computer. Running my own LLM locally is just for fun, and as of the time I write this confers no "practical" benefit to me to over using ChatGPT or Deepseek over the web.
## Docker Machine
Typically I run all of my containers on my "server" machine. I have a few related posts about this: [Server build pt. I](), [Server build pt. II](). However, running LLMs locally will require more powerful hardware than my server machine contains. Luckily my [other machine]() does contain powerful enough components, so I can easily run some smaller LLMs e.g. quantized and distilled models.
Typically I run all of my containers on my "server" machine. I have a few related posts about this:
- [Server build pt. I]({{< relref "posts/nas-build-pt-i">}})
- [Server build pt. II]({{< relref "posts/nas-build-pt-ii">}})
However, running LLMs locally will require more powerful hardware than my server machine contains. Luckily my [other machine]({{< relref "posts/pc-build">}}) *does* contain powerful enough components, so I can easily run some smaller LLMs e.g. quantized and distilled models.
I wrote a Docker Compose file that defines a service for running Ollama container, with support for GPU acceleration:
### Docker Compose File