As it feels like everybody has jumped on the local AI bandwagon by now, I felt a bit left behind. So it was time to dip my toes in the water and avoid my year-end admin chores. Of which the most mind-numbing is figuring out the tax mess, which in the Netherlands is almost entirely a snail-mail affair (yes, Dutchies, I am talking about business taxes). The amount of blue envelopes shipped to my home address is staggering and messy. A classic ‘unstructured data’ problem ripe for the plucking with the current state of local AI. It’s time to sort things out using some local LLMs and N8N to keep things private, secure, and as digital as possible.
First things first: I bought a Canon scanner (MF657CdW) with the ability to scan documents double-sided and store them fully OCR’ed on disk or cloud. Next step: add some RAM to my 5-year-old Desktop so it will happily run gpt-oss:120b. Update some broken Docker containers and finally determine why my port forwarding never worked (it sent the replies through my VPN). Port forwarding had to work to get OAuth flows running in N8N; the next blocker was my Authelia IAM, which needed some exceptions for call-backs to N8N.
Selecting the platform for running LLMs was easy: Ollama, as it had the option of opening up the LLM through an API. Installing N8N for agent/workflow tasks was also pretty simple (as I already had Postgres and Redis installed). Creating the workflow and connecting all the steps was a breeze. I successfully mailed my bookkeeper and uploaded the documents to their respective folders and bookkeeping software where necessary.
Lessons learned:
- N8N has some strange quirks around Oauth setup, first create the credential, and if the connection fails, try again via ‘in-private browsing’
- Filename changes when uploading documents are ignored, but using a bit of code to change the filename of the binary before uploading does work … Must be a n00b thing, don’t hesitate to give pointers.
- Let the LLM worry about its own shortcomings by feeding it its excrements and referring to the previously given spec. It took about 5 iterations before it created a verbose enough prompt that survived all the content (yes, even the butterfly my daughter drew for me, which ended up in the snail mail stack).
- Old hardware is still good enough for some fun, and when buying new, never skimp out on specs, especially RAM.
Next steps:
- When the document is a receipt or invoice, create a mutation in the right category in e-boekhouden.nl.
- Identify any actions and create Vikunja (my todo-list) tasks for them.
- Add all the data to a RAG DB in the hope the LLMs will get clever enough to actually help me optimize my tax strategies in the future (no, we’re not there yet). For now, I will hire Marieke for that.
But first: buy a lot more VRAM to speed up the larger models (8 tokens/s for gpt-oss:120b). When running in the background, this is an acceptable speed, but when debugging, I need things to run a lot faster.