Two-Tower Document Search

To rapidly deepen my expertise in deep learning, I enrolled in an intensive, eight-week bootcamp designed for experienced software engineers. One of my primary projects there involved building a semantic search engine using the MS MARCO dataset. 

Initially, I experimented with training custom word2vec embeddings on the text8 Wikipedia corpus. However, after careful evaluation, I opted for the reliable, open-source embeddings provided by Gensim.

The final architecture I implemented was a Two-Tower RNN model, featuring:

• Parallel encoding layers—one dedicated to queries and the other to documents (both positive and negative).

• Recurrent neural networks (RNNs) for capturing the sequential nature of text, preserving critical information such as word order.

• An InfoNCE loss function that encourages queries to be close to relevant documents while distancing them from irrelevant ones, thereby enhancing the overall quality of the learned representations.

This project served as a hands-on introduction to deep learning for semantic search and provided valuable insights into both embedding techniques and contrastive learning.