Berlin · 2026

Foudil Chebbour

I build Machine Learning systems, with special focus on quality evaluation, retrieval representation learning.

Research, released models, and production software I have worked on since 2022.

RoleSoftware/Machine Learning Engineer . Aristo Pharma GmbH.
ThesisHU Berlin · Prof. Lessmann / Akbik · submitted Sept 2026
Based inBerlin

StackPython · PyTorch · Hugging Face · Java/Spring · SQL · on-prem deployment

LENS · Evaluation framework

Framework + 2 encoders
Type M.Sc. thesis · Python package· Supervisors Prof. Stefan Lessmann · Prof. Alan Akbik, Humboldt-Universität zu Berlin·

LENS: text quality evaluation from four property-specific encoders

Logic, emotion, naturalness, semantics. Measured once by frozen encoders, combined per task.

Learned metrics and reward models often produce a single quality score, which can be difficult to interpret or adapt to a different evaluation task. LLM judges can provide explanations, but these need checking and add inference cost. In LENS, four frozen encoders measure separate properties of a text. A linear combiner with five parameters is fitted to human ratings for each task. Its coefficients and component scores show how each measurement contributes to the result. I evaluated this approach on translation, dialogue, and response preferences in RewardBench 2.

FROZEN · NO TASK SUPERVISION FITTED PER TASK candidate text anchor reference · prompt · context L E N S Logical relation Emotional content Naturalness of form Semantic content Off-the-shelf NLI encoder · entailment of anchor Our EmotionEncoder · cosine to anchor Our NatureEncoder · reference-free score Off-the-shelf Semantics encoder · cosine to anchor 4 scores Combiner Selected combiner by 1-SE rule over GLM, GLM + interactions EBM · GBM task quality score MT · reward modelling · dialogue

Fig 01Semantics, emotion and logic are scored against an anchor text (reference, prompt or context). Naturalness needs no anchor. The frozen part is identical across MT, reward modeling and dialogue.

Response preference evaluation: RewardBench 2 Focus, best-of-4

Evaluator How it judges Focus acc.
Claude 3.7 Sonnetzero-shot LLM judge92.1
Claude Opus 4zero-shot LLM judge86.2
LENS4 frozen encoders + lightweight linear combiner85.4
Gemini 2.5 Flashzero-shot LLM judge84.1
Gemini 2.5 Prozero-shot LLM judge80.5
Claude Sonnet 4zero-shot LLM judge76.0
GPT-4.1zero-shot LLM judge73.4
GPT-4ozero-shot LLM judge72.9

Accuracy in %, best-of-4: pick the human-preferred response among four candidates, 495 prompts. There is no gold reference, so each response is scored against its prompt. LENS is scored out of fold under 5-fold cross-validation grouped by prompt; the other rows are published leaderboard scores (allenai/reward-bench, June 2026). This comparison covers the Focus subset only, not the full benchmark. LENS uses task-specific preference labels during cross-validation; the listed LLM judges are zero-shot, so the training conditions differ.

  • A preference signal without an LLM at inference. LENS scores 85.4% in this setup, compared with published scores of 86.2% for Claude Opus 4, 73.4% for GPT-4.1, and 80.5% for Gemini 2.5 Pro. These are comparisons under the different training conditions described above. The four component scores make each prediction easier to inspect.
  • Relevance and entailment measurements for grounded dialogue. Semantic relevance of the answer to the query and entailment of the answer by the supplied context are LENS's semantics and logic scores. On knowledge-grounded dialogue (USR Topical-Chat), where a reply must use a supplied fact, entailment of the reply by that fact was selected in every fold for every rated quality; groundedness ρ = 0.43, mean ρ = 0.43 across the four shared dimensions against BARTScore 0.29 and UniEval 0.28, with LLM judges at ≈0.64.
  • Adaptation with a small fitted combiner. Adaptation fits five parameters while keeping the encoders frozen. In the translation experiments, 300–600 human ratings approached the performance obtained with ten times as many labels (a difference of ≈0.002 on the reported metric). This does not establish the label requirements for every domain. The encoders can run on-premises without an external judge call.
Machine Translation

The framework was built and first validated on translation, where the combiner is fitted on 6,000 WMT20 direct-assessment segments and tested on held-out expert MQM ratings. With five learned parameters it sits between chrF and COMET-22-DA at segment level (zh–en pairwise accuracy 0.495 vs 0.485 and 0.514) and close to COMET-22 at system level (0.79 vs 0.85 zh–en; 0.86 vs 0.89 he–en). The five parameters refer to the LENS combiner, not the full system: LENS also relies on pretrained encoders.

Released Emotion encoder Weights on Hugging Face
Backbone roberta-base-go_emotions · 128-d, ℓ2-normalised· Objective Weighted supervised-contrastive on GoEmotions·

EmotionEncoder

An encoder trained to make cosine similarity reflect shared emotional content.

Cross-entropy classification does not directly train cosine distances to reflect shared emotion. Low-dimensional affect representations offer another approach, but may lose distinctions between emotion categories. I wanted an encoder that ranks pairs by shared emotion and produces similarities that transfer across datasets. I trained it on GoEmotions with a supervised contrastive loss, weighting each pair by the overlap between its label sets.

Discrimination and calibration, in-domain and out-of-domain
Dataset Model ρ ↑ ROC-AUC ↑ Brier ↓ ECE ↓
GoEmotionsEmotionEncoder0.2340.7110.2770.421
GoEmotionsall-mpnet-base-v20.0260.5230.2760.416
GoEmotionsroberta-go_emotions0.2400.7160.3600.515
tweet_evalEmotionEncoder0.3630.7300.2550.273
tweet_evalall-mpnet-base-v20.1470.5930.2810.277
tweet_evalroberta-go_emotions0.2950.6860.3130.348

Target: do the two texts share at least one emotion label. GoEmotions (28 labels) is the training domain; tweet_eval/emotion (4 labels) is unseen. Brier and ECE treat (cos + 1) / 2 as a probability, same map for every model.

  • Under the fixed cosine-to-probability mapping, the semantic encoder has comparatively low calibration error but limited discrimination (AUC 0.52–0.59). EmotionEncoder improves discrimination over that baseline and reduces calibration error relative to the classifier backbone. These are relative improvements; the absolute ECE values remain substantial.
  • Brier score 23% lower than the backbone in-domain (0.360 → 0.277), on par with the semantic encoder.
  • The evaluation also found gains on two unseen label sets, with the widest positive–negative cosine gap among the tested models on each evaluated dataset. Clustering the emotion centroids gives a positive/negative split and groups like grief–remorse–sadness, with no loss term asking for that.
Released naturalness encoder + dataset
Backbone deberta-v3-base · 128-d, ℓ2-normalised· Objective Ordinal contrastive on an eight-level provenance spectrum· Release Weights on Hugging Face Dataset on Hugging Face

NatureEncoder and the naturalness spectrum

A reference-free score trained on an ordered spectrum of text sources.

Fluency judgments can be difficult to agree on, and perplexity is not a direct measure of naturalness. For this project, I built a dataset spanning older machine translation output, learner writing, and edited native prose, then trained an encoder on it. The dataset contains 65,868 sentences across eight levels, ordered using source provenance as a proxy for naturalness, rather than direct naturalness ratings; the encoder is trained with a contrastive loss whose pull falls off exponentially with the distance between two levels.

MACHINE TRANSLATION LEARNER ENGLISH NATIVE ENGLISH WMT06 fluency judgments Write & Improve, uncorrected OneStopEnglish 0123 4567 score 1–2score 3 CEFR ACEFR BCEFR C ElementaryIntermediateAdvanced 1,9472,560 6,9909,0997,786 8,1028,1958,015 severely disfluent edited native prose

Fig 02The eight levels with training-set counts. Older statistical MT output anchors the low end. The ordering is a modelling assumption: source, proficiency, and reading level are proxies rather than direct naturalness judgments.

Can the level be read off the embedding, and is it more than "which corpus"?
Encoder Linear probe ρ Pairwise cos. ρ within MT within Learner within Native
NatureEncoder0.9410.8910.0080.2220.577
Semantic (mpnet)0.6970.0660.0000.0130.004
EmotionEncoder0.582−0.0050.0060.001−0.010

Spearman ρ on the held-out split (13,174 sentences). Linear probe: ridge regression on frozen embeddings. Pairwise: cosine similarity vs distance between levels, over all pairs and over pairs from the same source corpus only.

  • A linear probe reads the level off the embedding at ρ = 0.941. Cosine tracks level distance at ρ = 0.891 (Pearson 0.909), supporting its use as a graded similarity measure on this dataset.
  • The eight levels come from three corpora, so the stricter test is within one corpus: ρ = 0.577 inside OneStopEnglish, where the same articles are rewritten at three levels, and 0.222 inside learner writing. The reference encoders have near-zero correlations in these within-corpus comparisons. NatureEncoder also has near-zero correlation within the MT subset (0.008), so the pattern does not hold uniformly across sources.
  • In LENS the score is the position between the level-0 and level-7 centroids, so no reference text is needed.

Production systems

4 systems + 2 backend applications

Built at Aristo Pharma GmbH since Nov 2022. The deployed systems run on-premises in a regulated pharmaceutical environment, using Java for integration and Python for ML. Implementation details are confidential.

SYS-01 · RAG

Domain RAG with a fine-tuned embedderShipped

Question answering over regulatory documents, supported by retrieval and answer-quality evaluation.

The corpus contains specialised vocabulary, abbreviations, and document structures that generic embeddings did not handle reliably in our evaluations. Fine-tuning on in-domain data improved retrieval; the system also combines sparse and dense retrieval.

Problem
Limited retrieval quality with generic embeddings, and a need for systematic evaluation.
Approach
In-domain fine-tuned embedder, hybrid retrieval, and an evaluation harness for retrieval quality, answer faithfulness and context relevance. Runs on SYS-04.
SYS-02 · ROUTER

Multi-LLM translation routerShipped

An on-premises translation application that selects a local LLM using empirical quality evaluations.

Confidentiality requirements called for local processing, and external API costs were another consideration. No single local model performed best across all evaluated inputs. For each model, I embedded the texts on which it performed best and averaged them into a centroid. The application uses distance to those centroids, alongside other criteria, to route new inputs.

Problem
Sensitive content, external cost, and no single best model.
Approach
Per-model centroids from empirical quality evaluation; routing on centroid distance and auxiliary criteria. Python for the ML, Java for integration.
SYS-03 · H-RAG

Hierarchical RAGIn implementation

Designed to split a query across source types, use an embedding model for each source family, and combine results in an embedded knowledge graph.

The design includes nested documents and multimodal inputs. Implementation is in progress.

SYS-04 · DATABASE

Regulatory document databaseShipped

System of record for regulatory and licensing documents; what the RAG system queries.

One of its inputs is an extraction pipeline that processes complete regulatory documents and writes structured records into the database. Java and PrimeFaces for the application, Python for the pipeline.

Problem
Regulatory documents with no structured, queryable record.
Approach
Structured database fed by a document extraction pipeline alongside the other inputs.

Other backend systems (Java · PrimeFaces, on-premises)

  • Document managementDocument editing and signing. Editing and signing regulated documents with identity-bound signatures, inside a larger document workflow.
  • Data governancePatient data management. Full-stack application for patient data: business logic, UI, reporting, case handling, subject to data-governance and security requirements.

Other research

1 project
Type Seminar · co-authored with Zhen Yang· Year 2025· Methods Semi- and self-supervised learning · Fairness · Calibration

Fair credit scoring under reject inference: autoencoder vs joint-embedding predictive architecture

A comparison of two latent-space approaches to selection bias, fairness, and calibration.

Credit models often learn from repayment outcomes for accepted applicants, while predictions are needed for all applicants. This creates potential selection bias; historical labels may also reflect discrimination, and predicted probabilities may be miscalibrated. We compared two approaches that address these concerns in a shared latent space: an autoencoder that pseudo-labels rejected applicants and puts a fairness penalty on the latent space, and a JEPA that predicts latents instead of reconstructing inputs.

Full-feature setting

Model AUC ↑ Brier ↓ F1 ↑ Eq.Odds ↓ Indep. ↓
JEPA + reject (988k)0.7030.1570.3690.0660.066
JEPA, no reject (988k)0.7070.1560.4180.5230.061
AE + reject0.7190.1580.3930.4030.048
XGBoost0.7030.1460.4290.6200.080
Unconstrained baseline0.7210.1440.4370.6500.200

Eq.Odds: equalized-odds gap. Indep.: independence (demographic-parity) gap. Lower is better for both.

How selection bias, fairness, and calibration enter the model

Proxy discrimination: an energy-based penalty pushes the latent distributions of the demographic groups together, so the sensitive attribute is not needed at inference time. Selection bias: rejected applicants get pseudo-labels from cosine similarity to applicants with known outcomes, in the same latent space the fairness penalty acts on. Calibration: heteroscedastic logits and a soft-ECE term during training, Platt scaling afterwards.

The JEPA variant avoids input reconstruction, which can encourage a representation to retain variation unrelated to default risk. A corrupted copy of the input is encoded and trained to predict the latent of the clean copy, with VICReg terms against collapse.

Earlier work

2 projects
  • Bilirubin measurement from digital images2022 — estimate bilirubin, melanin, and haemoglobin concentrations from the RGB camera response using the Beer–Lambert law; ρ = 0.976 against blood tests.
  • Acoustic quality index for pharmaceutical pelletsB.Sc. thesis, 2022 — an index from sound speed that predicts the best water content after the first process step instead of the third; 94.7% across 7 formulations.

Contact

Email
foudil.chebbour@gmail.com
GitHub
github.com/foudil09
LinkedIn
Foudil Chebbour
Hugging Face
huggingface.co/foudil