Zeu Park

A curious developer documenting my journey through code, from Python basics to machine learning and beyond

2 minute read

Background

While working on an AI service project that involves vector similarity search, I had to choose between two widely used vector database options: FAISS and Weaviate. Both offer solid functionality, but their system requirements and architecture made me rethink what’s feasible given my current development environment.

This post outlines the technical and practical trade-offs I considered, the issues I encountered, and why I ultimately chose FAISS.

Initial Considerations

At first, Weaviate appeared to be a compelling option:

  • Built-in RESTful APIs
  • Modular plug-ins
  • Scalable architecture
  • Integration with OpenAI, Cohere, and others

However, the Docker-based deployment of Weaviate introduced significant overhead. It consumed more RAM and CPU cycles than I could afford on my current machine setup, especially since I needed to reserve capacity for the model serving pipeline.

I had discussed this with ChatGPT earlier and realized that while Weaviate provides many “convenient” features, they come with a price—namely, increased system complexity and resource usage. This wasn’t ideal for a CPU-based prototype or tight resource constraints.

Decision Pivot

The conversation led me to reconsider FAISS. Its advantages:

  • Lightweight and fast
  • Offline index generation
  • Works well in CPU-only environments
  • High customizability with manual control

Despite requiring more setup code and lacking built-in APIs, FAISS fit my immediate goal: to get the core model serving reliably working on limited hardware.

My key realization was:

It doesn’t matter how fancy your vector DB is if your model can’t even run.

Final Choice: FAISS

After several test runs and comparisons:

  • Weaviate: Dockerized deployment worked but consumed too many resources.
  • FAISS: Simple Python integration, low memory usage, and fully manageable from my FastAPI backend.

I was able to integrate FAISS with my retrieval pipeline, using MiniLM-L6-v2 embeddings and a basic cosine similarity search via FAISS’ flat index. This was enough to build the first working prototype of my LLM service.

What I Learned

  • Realistic constraints matter. An architecture that works in theory may fail in practice if you don’t have the infrastructure to support it.
  • Simplicity scales better when you’re learning. FAISS forced me to understand how vector search works under the hood, and that knowledge is now reusable.
  • Don’t over-engineer too early. Weaviate might be a better choice later, but not at the prototyping stage on a limited machine.

Next Steps

  • Explore hybrid indexes in FAISS for better recall
  • Eventually try Weaviate again on a cloud GPU instance
  • Document performance benchmarks between FAISS and Weaviate in controlled conditions

You May Also Enjoy