Chunk Size

• Searching for Best Practices in RAG:

Chunk size significantly impacts performance. Larger chunks provide more context, enhancing comprehension but increasing process time. Smaller chunks improve retrieval recall and reduce time but may lack sufficient context.

Generally, the chunk strategy and the size have to be fine tuned for specific situation, but the paper Searching for Best Practices in RAG provides at least some guide what should be good starting chunking size of 256 with overlap of 20 tokens: