From 81dc8bdcbedbb7b8e6aed1fe108f48e8315816e2 Mon Sep 17 00:00:00 2001
From: leduc <leduc@MBP14.local>
Date: Sat, 15 Mar 2025 12:12:40 +0100
Subject: [PATCH] results consistency explanation

---
 .DS_Store | Bin 6148 -> 6148 bytes
 README.md |   5 ++++-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/.DS_Store b/.DS_Store
index 870c31a89a45d8542d4adaf6363aaf8e725a78eb..96bed275d6f891f5ed16243b0b39b8e09424fd1a 100644
GIT binary patch
delta 111
zcmZoMXffEJ#u9gG76StV3xgg*IzuKyNp8N2OHxjL5>SjoK-9$c^r2&psPZXz<qI+l
agOl@f3xIkU7<ex*OwMCf+q{Y;N*Dkk3LDb^

delta 111
zcmZoMXffEJ#uB%49s>gd3xgg*IzuKyNp8N2OHxjL5>Sl8N%Up*sYAycQRP$M@-ytj
gGSKA<G7N*0^K%P;dKegZCooLTV^!O{iX}=I0E+G%R{#J2

diff --git a/README.md b/README.md
index 0eb7767..e436279 100644
--- a/README.md
+++ b/README.md
@@ -1,4 +1,4 @@
-# Codebench - Ollama Model Benchmark Tool
+# Codebench - Ollama Models Python Benchmark Tool
 
 A Python-based benchmarking tool for testing and comparing different Ollama models on coding tasks. This tool allows you to benchmark multiple Ollama models against common coding problems, measure their performance, and visualize the results.
 
@@ -76,6 +76,9 @@ The tool currently tests models on these coding challenges:
 1. Each model is prompted with specific coding tasks
 2. Generated code is extracted from the model's response
 3. Initial syntax validation is performed
+4. Code that fails validation is passed to Together API for advanced code analysis
+5. Code that passes validation is executed and validated with given data and compared to expected results
+6. Each test is run 4 times for consistency and only the last 3 results are used for metrics
 
 ### Test Validation
 For each test case: