Helicone

Helicone is a company developing an open-source observability platform for generative AI. Helicone offers insights into a company's generative AI usage, logging requests made to providers like OpenAI to monitor spending, analyze traffic peaks in order to allocate resources more efficiently, and track latency patterns. As a low-latency proxy server, Helicone sits on top of wherever inference requests are called, adding negligible latency while logging the full state of the request and response for any endpoint. Users can optionally add application-related metadata to these requests for analytics.

Helicone provides tools to control access to the user's system with the following:

• User rate limiting—limits the number of requests per user
• Metrics—identifies power users and optimizes the application for them
• Request retries—retries failed requests

And tools to scale, manage, and control AI applications:

• Bucket Cache—reduces costs by caching and configuring responses
• Custom Properties—tags requests to segment and analyze traffic
• Streaming Support—analytics into streamed responses out of the box

Helicone offers a fully-managed cloud solution and deployment on AWS.

Helicone is being developed by Scott Nguyen, Barak Oshri, and Justin Torre (CEO) with work starting in December 2022. Oshri brings machine learning experience, researching, and teaching at Stanford AI Lab and Sisu Data. Nguyen has worked in UX and finance with experience at Tesla, Bain Capital, and DraftKings. Torre has platform and full-stack expertise working at Apple, Intel, and Sisu Data. Based in San Francisco, the company took part in Y Combinator's W23 batch, launching on February 23, 2023.

Helicone is free to start using with pricing plans based on monthly requests. The basic flex plan offers 100k free requests a month, with a pay-as-you-grow model, charging $1 for every additional 10k requests. Helicone also offers an enterprise plan designed for large organizations offering custom request limits, advanced features, and 24/7 expert support.