Freelancer Spotlight - AI in Finance
AI-powered financial intelligence and analytics
Dec. 3, 2023
LLMs are at the forefront of a computing revolution, seamlessly integrating into various industries. Our fascination with these advancements grows as we learn more about the diverse applications of LLMs from innovators in the AI community. One such innovator we recently got to know is Othmane Zoheir. His journey, starting from high-frequency trading on Wall Street to founding rumorz.io, an AI-powered financial intelligence and analytics platform, is very interesting.
In this post, Othmane shares the motivation and purpose of building Rumorz, and how LLMs are helping achieve that.
Background
Please introduce yourself briefly—your educational and professional background and how you got involved with AI.
My name is Othmane, and I am a quant trader and builder of AI, trading and fintech applications. Trading, technology and AI have been my main interests since high school.
My first job out of school after graduating from the University of Michigan undergrad in 2015 was in high frequency trading on Wall Street, where I traded all day and built profitable FX market making algos for a top bank. I was having fun trading, coding and actively participating in global markets.
After 2 good years, I was even more fascinated by automation and intelligence so I went to back grad school in 2018 to study machine learning and NLP.
I’ve been working independently since then, building tech and trading applications for hedge funds and clients, and recently founded rumorz.io, an AI powered financial intelligence and analytics platform.
Though I’ve been using NLP and transformer based models since 2018, I went deep into generative AI early 2023. We’re clearly at the beginning of a new software paradigm so I thought now is the time to go all in on building a product!
During my LLM journey, I got a bit fed up with frameworks like langchain and llama-index, so I’ve built an open-source LLM library called tinyllm which I use in every LLM project of mine and in Rumorz! The library is still growing and is powering my apps in prod.
I keep up to date with the latest AI news, papers, models and tools releases, but also focus on building around what is deployable in production today. LLMs need a domain expert to really add value so I try to focus more on the data side than the generative side.
Your Project Portfolio
In your freelancing experience, what types of business problems have you encountered where businesses see value in adopting LLMs?
Financial intelligence
My main use case for LLMs is building Rumorz. The main benefit of LLMs for the app is in organizing financial data in a knowledge graph. LLMs excel at NER and semantic understanding so a financial knowledge graph immediately came to mind when LLMs became cheap to use. A Knowledge graph will allow traceable retrieval and Q&A to chat with the real-time graph, but also running models and alpha signals on the knowledge graph.
Stock options assistant
I recently built a Q&A chatbot to answer questions related to Stocks Options data for an Options Trader. The idea was to build an app to allow any Options trader to get fast answers around prices, volumes and metrics from the options market.
Ecommerce and customer support
I got into LLMs through a conversation with a friend who owned a Shopify store and was discussing the benefits on an LLM based chatbot. The same evening, I got into langchain, llama-index, vector stores and such to build an MVP chatbot for the store, available on Github. The chatbot was a langchain RAG with access to the store’s products, inventory and FAQ and it blew her mind. I realized how powerful LLMs were while building this chatbot.
tinyllm
After building 1 to 2 chat applications, I decided do build a leaner Python LLM library than langchain, llama-index etc which come with large unnecessary overheads.
The library is open-source and I’m using it for all my LLM apps! Building it not only helped me simplify my LLM development, but also go deeper into LLM ops and what the infratructure requirements were to build scalable LLM applications.
Tools of the Trade
What are your preferred tools for building LLM applications? Any go-to models, orchestration/observability/evaluation frameworks?
Like many others, I’ve started exploring RAG applications with langchain and llama-index. Prototyping with those tools was great, but I was often facing issues with code complexity, debugging and lack of documentation.
After months of using different tools, I’ve developed my own open-source Python infrastructure called tinyllm, which includes chains/RAG development, observability/tracing and evaluation. The idea is to keep code simple, easily traceable and scalable. It can integrate with any other LLM libraries like langchain if desired. I use tinyllm to develop Rumorz and it’s been stable for a while now!
In terms of chat models across, my go-to is still OpenAI’s gpt 3.5, and gpt-4 for very complex tasks. With that said, I like Mistral 7b and other smaller open-source models.
For production use, I think lots is doable with open-source, provided you know how to create good prompts or finetune models and have the budget to deploy your own model instance.
For embeddings, I’ve moved away from OpenAI’s da-vinci as it creates an unnecessary bottleneck in a pipeline due to rate limits and costs. I’ve instead for open source embeddings models as they’ve proved to be faster, free, and sometimes even more accurate than OpenAI’s embeddings.
ProjectsProject 2: Rumorz.io - the real-time financial knowledge graph
Project 1: Stock options agent
Overview: What is the project about? Who was the client (company size or sector)?
A Stock Options trader needed a chatGPT like Options agent to answer any questions related to Stock options: Pricing, Volumes, Order flow, Greeks, Pricing and Strategies
What specific business problem was the project aiming to solve?
The problem with navigating options is that access and readability of Options data is complex and not easily implementable. For each stock ticker, there are often 100s of options contracts available for pricing, trading and analyzing (different strike prices, maturity etc…). Today, traders cannot easily price and get live data on any given stock options strategy and flexible/sophisticated options pricing tools are expensive.
Technical Implementation:
- How did you integrate the LLM into this project?
LLMS are used for the generation step of a RAG pipeline powering a Stock options agent. - What were some technical challenges you faced, and how did you overcome them?
The main challenge was finding the right data architecture to allow easy communication between our database and the options agent. On one hand, we need to keep the database in sync in real-time through ETL jobs, on the other hand the agent queries the DB. GPT-4 is very good for generation so the main challenge was setting up the right retrieval pipelines.
Measurable Outcomes:
- Any success metrics you can share to demonstrate the impact of the LLM on this project?
All potential questions about options provided by the client were answered successfully. The client can now gain real-time advanced knowledge in a matter of seconds. - Any client testimonials or data points that validate the success of the project
The client was satisfied with the chatbot’s accuracy and speed, and wants to build on top of it so that is likely a good sign of success!
Project 2: Rumorz.io - the real-time financial knowledge graph
Brief Overview
This is actually a personal project I have been working on the side for years. It started as trading tools and algos I developed on my free time for my personal trading account. I’ve recently decided to productize all those tools and ideas and make it an app, rumorz.io
What specific business problem was the project aiming to solve?
Over my 10 years working in institutional trading and investments, I’ve spent hundreds of hours per year reading news, compiling summaries and extracting narratives and insights from financial news. Either for trading decisions, client reports or investment thesis development. Specifically, the business problem with financial news is 2 fold:
The solution is a real-time financial knowledge graph coupled with an AI copilot to access, understand and analyze financial markets in real-time.
Technical Implementation:
- How did you integrate the LLM into this project?
Rumorz uses LLMs in 3 ways:- To build a financial knowledge-graph from real-time financial data
- Data and ML jobs like topic modeling, monitoring and semantic understanding
- Agent with Retrieval Augmented Generation access
- What were some technical challenges you faced, and how did you overcome them?
- The first challenge was definitely building the real-time knowledge graph engine. This required real-time listening+indexing of all kinds of financial data (tabular, text, tables, PDFs…) and binding that with a graph database layer. This required lots of prompt engineering, validation layers + thorough testing of many edge cases to make sure the graph can be updated reliably in real-time in a production environment.
- The second challenge was building a versatile RAG pipeline from the knowledge graph, allowing Q&A on as many questions on possible, both about financial news, but also tabular and analytics data. Similarly, this required lots of prompt engineering, evaluation and modeling. Fortunately, I have managed getting the RAG to a pretty good place now and can’t wait to get it out to users.
The knowledge graph: each node represents a financial asset, person, company, country etc, and every edge between nodes represents a semantic relationship between the 2 entities. When a user asks a question, the RAG pipeline will find the relevant subgraph as context to answer
The rumorz.io app: rumorz extracts sentiment, emotion and trending themes from the financial knowledge graph in real-time.
Measurable Outcomes:
- Any success metrics you can share to demonstrate the impact of the LLM on this project?
As of today, the real-time knowledge graph has been running in production without any bugs or errors for a couple of days. Financial news and data is being ingested in real-time and am getting ready to launch the app! - Any client testimonials or data points that validate the success of the project?
As the first user of the app myself for trading and staying up to date with markets, I am very happy with its capabilities as it helps me understand, in 5 minutes, what’s happening in the markets without getting overwhelmed and clicking on 10 different articles and PDFs.
With that said, the only real validation will be revenue once the app launches!
Your Services and Offerings
Please share details about the services you offer related to LLMs. This is your opportunity to pitch your services to potential clients. Also include feel free to include your contact info/portfolio/LinkedIn etc.
We offer AI, chatbot and ML application development across all industries: finance, marketing, Ecommerce, HR, real estate etc, for startups, SMB or larger corporates.
We start from the client’s problem and current solution to the problem, and from there, reverse engineer a solution tailored specifically to the client’s existing tech stack, operations, UI/UX and level of tech proficiency. This typically includes includes development, training and education for user adoption.
Below is my portfolio and social links!