In one of our earlier articles , we touched on statistical metrics and how they can be used in evaluation - we also briefly discussed precision, recall, and F1-score in our article on benchmarking . Today, we’ll go into more detail on how to apply these metrics more directly, and more complex metrics derived from these that can be used to assess LLM performance. This is a standard measure in statistics, and has long been used to measure the performance of ML systems. In simple terms, this is a measure of how many samples are correctly categorised (true positives) or predicted by a model out of the total set of samples predicted to be positive (true positives + false positives). If we take a simple examples of an ML tool that takes a photo as an input and tells you if there is a dog in the picture, this would be:

In our previous article , we talked about the highest level of testing and evaluation for LLM models, and went into detail about some of the most commonly used benchmarks for validating LLM performance at a high level. Today, we’re going to look a at some more fine-grained evaluation metrics that you can use while building an LLM-based tool. Here we make the distinction between statistical metrics - that is those computed using a statistical model - and more generalised metrics that attempt to measure the more ‘subjective’ elements of LLM performance (such as those used in manual testing) and that use AI to evaluate how useful a model is in its given context. In this article we’ll give an overview of the different classes of metrics used and cover human evaluation and its importance before moving on to common statistical metrics and LLM-as-Judge evaluations in the following articles.

Welcome! This is part 5 of our course on how to build fullstack apps with Bolt and Supabase If you’re just joining, I highly recommend you take the course in the correct order before diving into this one. Here you can find Part 1 , Part 2 , Part 3 , and Part 4 .

In this article, let’s continue to explore Cursor even further. Our first article ( which you can find here ) covered Cursor’s basics and the easiest-to-understand features, such as Rules for AI , Tab autocompletion and the Chat feature. So, if you’re new to Cursor, I highly recommend you check the previous article. In this “Part 2” article, we’ll go over the Cmd+K , Composer , and Agent features, including some use cases. So, get ready to learn how to use Cursor to its fullest potential and save an enormous amount of time. Starting from version 0.46, Cursor includes a lot of UI changes for the AI side panel. That’s why, if you’re currently using an older version, the UI elements mentioned in this article might not look the same for you. That’s completely fine, but I highly recommend you update to the latest version so we’re on the same page.

If you’re into AI agent development, you’ve probably started hearing more and more about a new emerging protocol – Model Context Protocol ( MCP ). In essence, this protocol simplifies how AI agents connect to the data and tools they need. By standardizing these connections, MCP reduces the extra work developers usually have to deal with. Essentially, it replaces the need to directly manage multiple APIs in your AI agent with one unified protocol. And lets you to add/remove any external tools for your agent with incredible ease. Making it more convenient to build complex and flexible AI systems. In this article, we’ll walk you through everything you need to know about MCP—from its core components and main concepts to practical implementations. We will focus specifically on building an MCP server, as it is likely the most useful and frequently used part of the MCP architecture that you will want to implement. So, let’s go! The Model Context Protocol, or MCP, is a simple standard, designed and open sourced by Anthropic to help AI tools talk to the systems where data lives. Think of it like a USB-C port, but for AI applications. Just as a USB-C port lets you connect different devices with one common plug, MCP lets AI models easily connect to various data sources and tools without needing custom code for every connection.

Everyone knows chatgpt, but how do modern large language models fully work? The fundamentals start at the transformer. This workshop is a workshop to dymstify the transformer and be able to run through concept to code on how the transformer work. This workshops combines concept at an intutive level, to code, to math all with the intent at providing an end to end understanding at the fundamentals of large language models.

<!-- Your template is like this (markdown): --> Learn the strategies senior engineers use to configure their projects to run securely, anywhere with environment variables. <!-- ( the summary is short - one or two lines ) -->

Large Language Models are reshaping industries, yet integrating them into real-time streaming UIs presents unique challenges. In this course we will learn how to seamlessly integrate LLM APIs into applications and build AI-powered streaming text and chat UIs with TypeScript, React, and Python. Step-by-step, we will build a full-stack AI application with quality code and very flexible implementation. The LLM application in this course includes: - Completion Use-Case (english to emojis) - Chat - Retrieval Augmented Generation use-case - AI Agent Use-Cases (code execution, data-Analyste agent) This app can be used as a starting point in most projects, saving a huge amount of time, and its flexibilty allows new tools to be added as needed. At the end of this course, you will have mastered end-to-end implementation of a flexible and high-quality LLM application. This course will also equip you with the knowledge and skills necessary to create sophisticated LLM solutions of your own.

This course will teach you how to fetch in a performant way for React without relying on a third-party library or the useEffect hook. We will build a custom data fetching library from scratch that will prefetch, cache, and address all the common issues with fetching data.

Learn Pro Patterns for Hooks, Testing, Redux, SSR, and GraphQL

Practical Security for Busy People

Learn Data Structures and Algorithms in JavaScript

A Field Guide to Your New Career

The Complete Guide to Developing Data Visualizations with D3