Getting Started with Nvidia AIPerf tool for LLM Performance testing

Getting started with Nvidia AIPerf

The new Nvidia AIPerf tool is an excellent free tool for LLM Performance testing. You can customise it as per your needs and is a massive upgrade to other tools especially if you use Nvidia GPUs.

How to Reduce CPU Spikes for AI Summarization

Reduce CPU spikes - AI Summarization

Summarization aims to compress a lengthy source document into a concise format while retaining its core components and key ideas.

However, when you are hosting your own LLM, handling CPU spikes (in the absence of a GPU) can be your biggest concern.

How to enable NVIDIA GPU workloads on k3s cluster

 

GPU workloads on k3s

K3s is a highly available, certified Kubernetes distribution designed for production workloads. It can also be used for AI workloads.

 By default, k3s nodes do not recognize GPUs. In this article, we will enable k3s to work with a GPU. 

How to deploy Google's latest Embedding Model embeddinggemma-300m on Nvidia Triton Server

EmbeddingGemma on NVIDIA Triton Server    

An embedding model is needed to generate embeddings (vector representations that help an LLM to understand things like text, images etc). 

Google recently released EmbeddingGemma-300M (a whooping 300 million parameter model) which has very low requirements to run. We test this assumption in this article.

Getting Started with Cloudflare's new AI Crawl Control

 

 

AI Crawl Control From Cloudflare

  

  Cloudflare has recently announced a new feature  called AI Crawl Control . 

  

 

How to do a Performance Test for AI Models hosted on NVIDIA infrastructure?

 

 

A performance test is extremely important for an Artificial Intelligence (A.I.) Model just as it is important for an e-commerce website like Amazon.

Getting Started with NVIDIA Triton Server

 

NVIDIA Triton Inference Server

What is an Inference Server?

The role of an Inference Server is to accept user input data and pass it to an underlying trained model in the required format and return the results. It is also widely known as a Prediction Server as the results are nothing but predictions (in most cases).

The NVIDIA Triton server is a gold standard that standardizes AI model deployment and execution across every workload and it is important to know how it works internally for your custom or off the shelf models.