Testing and Evaluation: Our Protocol for Espresso Machine Reviews

Why a standardized test protocol?

When buying an espresso machine, you face an overwhelming selection: From the compact Thermoblock machine for 400 euros to the dual boiler monster for 4,000 euros – how do you compare? That's exactly where our test protocol comes in.

We don't just want to tell you "This machine is great" or "That one is bad." Our goal is to provide an objective, measurable, and comparable assessment based on hard facts. Here, the result in the cup is front and center, supplemented by measurable technical parameters and a fair economic classification.

All results are rated on a scale of 0 (insufficient) to 10 (outstanding) and then weighted into an overall score.

Overview of all espresso machines we have tested.

How you should read this protocol

1. Look at the overall score first – it gives a quick overview of overall quality
2. Identify your personal priorities – What matters to you? Espresso potential? Steaming performance? Energy efficiency?
3. Focus on the relevant individual scores – Ignore categories that are unimportant to you
4. Compare only machines in similar price ranges – A score of 7.5 at 800 euros can be better than 8.0 at 2,500 euros

The weighting: Not all criteria are equally important

To do justice to the different relevance of the aspects, the categories are weighted with different factors in the overall rating:

Factor 3 (High – The Core)

Espresso potential (3×): The most important criterion. How good is the espresso that can be extracted at its best from the machine? Technical prerequisites merge here with sensory validation.

Factor 2 (Medium – The Technique & Haptics)

These categories significantly influence the daily experience and technical performance:

• Temperature management (2×): Consistency and adjustability
• Volumetrics (2×): Precision of automatic portion programming
• Steaming quality (2×): Performance and speed when steaming
• Workmanship (2×): Materials, build quality, and durability
• Usability (2×): Workflow, cleaning, and ergonomics
• Heat-up time (2×): Time until fully ready for use
• Power consumption (2×): Energy efficiency during operation

Factor 1 (Low – Supplements & Special Categories)

These aspects are relevant but less decisive for most users:

• Noise level (1×): Sound emissions during operation
• Accessories (1×): Quality and scope of included items
• Volume potential/Catering (1×): Performance capability with multiple consecutive shots
• Price-to-performance ratio (1×): Economic classification

Overall score calculation

The sum of weighting factors is 21: once factor 3, seven times factor 2, four times factor 1. The result is a score between 0 and 10.

The quality classes

Depending on the overall score, we assign the machine to a quality class:

The heart of our test

Why this category is most important

In the end, only one thing matters: How does the espresso taste? A machine can be technically impressive, but if the result in the cup disappoints, everything else is secondary. That's why espresso potential carries Factor 3 in the overall rating – more than any other category.

Our hybrid approach: Technique + Sensory

Espresso potential is derived from two pillars:

• 1. Technical prerequisites (max. 8 points) – objectively measurable
• 2. Sensory validation (max. 2 points) – panel testing

The technical prerequisites (8 points)

A. Temperature performance (0-3 points)

The score from Chapter 6 (Temperature management) is used as the basis here:

• Score ≥ 9.0 in Chapter 6 → 3.0 points
• Score 8.0-8.9 → 2.5 points
• Score 7.0-7.9 → 2.0 points
• Score 6.0-6.9 → 1.5 points
• Score 5.0-5.9 → 1.0 points
• Score < 5.0 → 0.5 points

Temperature stability is the foundation for good espresso. A machine with excellent temperature management already has a major advantage.

B. Profiling & brew control (0-3.5 points)

Manual control (0-3 points):

• 3.0 pts: Flow + pressure control: everything goes. Formulation of if-conditions, stages, volumetric control
• 2.5 pts: Programmable pressure profile control
• 2.0 pts: Adjustable pre-infusion (time/pressure) OR simple pressure profiles
• 1.5 pts: Fixed (sensible) pre-infusion present
• 1.0 pts: Manual profiling via paddle/lever possible
• 0 pts: No profiling options

Smart profiling bonus (+0.5 point):

Some modern machines have adaptive systems that monitor flow during extraction and automatically intervene. If the machine detects, for example, a too-fast flow (indicator of too coarse a grind), it reduces pressure or extends the pre-infusion automatically. This greatly increases error tolerance and enables even less experienced users to consistently produce high-quality results.

Maximum: 3.5 points

C. Hardware hygiene (0-2 points)

Shower quality & water distribution (0-1 point):

• 1.0 pts: High-quality shower, high-quality sieves, uniform distribution
• 0.5 pts: Solid standard shower, sieves of medium quality
• 0 pts: Poor distribution, channeling susceptibility

Thermal mass & pressure build-up (0-1 point):

• 1.0 pts: Heated-through metal basket, stable pressure build-up
• 0.5 pts: Partial thermal mass or slight pressure fluctuations
• 0 pts: Cold basket or unstable pressure

The sensory validation (2 points)

Technical values are the foundation – but can you taste that?

Panel test by Q-Grader Benjamin Hohlmann with reference coffees whose ideal profile is known.

Assessment:

• +2 points: Machine fully leverages technical potential. Espresso shows balanced harmony, no extraction errors
• +1 point: Technical potential is largely leveraged, minimal deviations
• 0 points: Clear discrepancy between technique and cup result

We rate the machine's ability to produce fine-pored, homogeneous microfoam for latte art. This assessment is made by our barista professional Michel Indelicato, who has steamed with around 100 different machine types in test scenarios.

The categories (max. 10 points)

A. Steam power

Pressure and ability to spin 300ml of milk in a 0.6l pitcher. The steeper a pitcher must be held, the less powerful the machine.

• Pitcher vertical (0°) → Strong power
• Slightly angled → Good power
• Steeply angled → Weak power
• No rotation → Insufficient

B. Speed

Time required to heat 300ml of milk from 8°C (refrigerator temperature) to 60°C. In this scenario, milk is deliberately not steamed but only heated (steam wand inserted deep).

C. Workflow & ergonomics

• No-burn steam wand or insulated design
• Ball joint (360° freely movable)
• Rotary valve (precise steam quantity control) vs. simple toggle switch

D. Condensation

How much condensation or residual moisture does the machine discharge before steaming begins? Too much condensation waters down the milk.

The robustness test

We test not just whether volumetrics is present, but how robustly it works under varying conditions.

The 5-stage stress test procedure

Setup:

• Coffee: Apas Espresso (medium roast)
• Grinder: Mahlkönig E64 GbW
• Brew ratio: 1:2.5
• Target size: Depends on basket size, remains constant

The 5 stages:

1. Hardware basis (5×): Scace basket (simulated resistance) – testing pure technique without coffee puck influence
2. Reference espresso (5×): Real conditions with standard grind
3. Profile check (5×): Test with complex profiles (e.g., pre-infusion) to check if logic remains stable with pressure fluctuations
4. Stress test "coarser" (3×): Grind is set one full number coarser (faster flow)
5. Stress test "finer" (3×): Grind is set one full number finer (slower flow)

Goal: Does the machine detect the changed flow (flow meter) or does it stubbornly stop after time?

Scoring grid (0-10 points)

Base points (technology type):

• 0 pts: Manual (no volumetrics)
• 3 pts: Pure time control

Precision points (based on average deviation):

• 4 pts: ±4g deviation
• 6 pts: ±2g deviation
• 8 pts: ±0.5g deviation (scale level)

Bonus (+2 points max):

• +1 pt: Exact UI input (grams/milliliters instead of pulses)
• +1 pt: Integrated scale

With scale coupling, we run the protocol twice: once without, once with scale.

This category rates how pleasant the machine is to use daily. A technically brilliant machine that annoys you every day loses points here.

Point distribution (max. 10 points)

• Interface & Control (max. 3 points + 0.5 bonus)
• Daily routine (max. 3 points)
• Cleaning & maintenance (max. 3 points)
• Documentation (max. 1 point)

A. Interface & Control (0-3.5 points)

For machines WITH display/software:

• 3.0 pts: Intuitive, no manual needed, quick response time (<0.5s), clearly readable
• 2.5 pts: Good, short learning curve, display still readable in sunlight
• 2.0 pts: Solid, occasionally manual needed, acceptable readability
• 1.5 pts: Cumbersome menu navigation, frequent reference to manual needed
• 1.0 pts: Confusing, sluggish response, poor readability
• 0.5 pts: Frustrating to operate

For machines WITHOUT display (mechanical/analog):

• 3.0 pts: High-quality controls, clear tactile feedback, precise rotary dials
• 2.0 pts: Solid haptics, functional
• 1.0 pts: Wobbly feel, imprecise adjustment
• 0 pts: Cheap plastic, significant play

Bonus: +0.5 pts for programmable shortcuts/function buttons

B. Daily routine (0-3 points)

Each met criterion gives 0.5 points (max. 3):

• ✓ Water tank easy to remove/fill without jamming
• ✓ Water level always clearly visible or last shot technology
• ✓ Drip tray adequately sized (>500ml) and easy to remove
• ✓ Basket smooth-running, secure locking without force
• ✓ Integrated water filter in tank present
• ✓ Ergonomic arrangement of all daily-use elements

C. Cleaning & maintenance (0-3 points)

• 3.0 pts: Automatic cleaning programs available AND shower easily removable with spoon or supplied tool
• 2.5 pts: Automatic programs available, partial disassembly tool-free, tool not in package
• 2.0 pts: Easy manual cleaning, clear machine prompts for maintenance intervals
• 1.5 pts: Manual cleaning possible, somewhat fiddly
• 1.0 pts: Cumbersome disassembly, unclear maintenance hints
• 0.5 pts: Hard-to-access parts, no maintenance hints

D. Documentation (0-1 point)

• 1.0 pts: Excellent manual (clearly structured, good imagery, complete)
• 0.5 pts: Acceptable manual with minor gaps
• 0 pts: Poor or incomprehensible manual

Penalty: -1 point for significant "annoyance factors" (excessively loud beeps, unnecessarily complex procedures, long unexplained waiting times)

The technical foundation

Why temperature is so important

Brew temperature is one of the most critical parameters in espresso. Just 2°C difference can drastically change the taste: too cold = sour and underextracted, too hot = bitter and burnt. We don't rely on manufacturer specifications; we measure ourselves – under standardized conditions.

Our measurement setup

Temperature measurement:

• Device: Scace 2 with data logger
• Calibration: Before each test series with boiling water
• Measurement: Directly in the basket

Power measurement:

• Device: Christ CLM1000 Professional (Plus)
• Resolution: 0.1 Wh (0.0001 kWh), second-by-second readings
• Logging: Saved as CSV file

Environmental conditions:

• Water temperature in tank: 20°C (±2°C)
• Target temperature: 93°C

The 93-degree standard and offset calibration

Our reference value is a brew temperature of 93°C.

With offset: If the machine has adjustable temperature control (PID), we calibrate the offset so that the machine in stable state (shots 3-5) delivers an average of 93°C as precisely as possible.

Without offset: If temperature cannot be adjusted, we measure the machine in its factory settings.

When is a machine "heated up"?

Manufacturer specifications are often optimistic ("Ready after 5 minutes"). We define the status "heated up" more strictly:

Example: If the machine settles at 92.5°C after an hour, it must deliver at least 91.5°C on the first shot after heat-up.

Special case: With light baskets (plastic/silicone insert), we allow a 1.5-degree deviation, as they work thermally differently than our measurement basket.

We determine actual heat-up time using a cold-start procedure in 5-minute intervals (5, 10, 15, 20 min) until this criterion is met.

The two test protocols

The KM Protocol – The everyday test

Simulates typical home use: After heat-up, we pull 5 double espressos at one-minute intervals.

Procedure:

1. Insert Scace
2. Start shot: Extract 27 seconds
3. Dwell time: Keep Scace locked for 15 seconds
4. Unlock
5. Rest: Let Scace sit unlocked for 45 seconds
6. Flush: 2 seconds flush
7. Repeat (total 5 shots)

Goal: Check if the first shot is solid (cold-start performance) and if the machine maintains temperature over a series.

The WBC Protocol – The stress test

Per World Barista Championship rules, the machine must perform under fire: 14 shots in quick succession plus parallel milk steaming. This shows whether the machine has reserves for guests or café use.

The WBC protocol is conducted after one hour of running time.

The 3-cluster scoring model (max. 10 points)

Cluster A: KM everyday suitability (max. 5.3 points)

A1. Cold-start behavior / First shot (max. 3.3 points)

Comparison: Average of shot 1 vs. average of shots 3-5 (steady-state)

• Difference < 0.5°C → 3.3 points
• Difference < 1.0°C → 2.7 points
• Difference < 1.5°C → 2.0 points
• Difference < 2.5°C → 0.7 points
• Difference > 2.5°C → 0.0 points

A2. Consistency & calibration (max. 2.0 points) – FAIRNESS UPDATE

Check stability of shots 3, 4, and 5 relative to each other.

Scenario A (Stable): Delta of averages < 0.5°C?
→ 2.0 points (Full points for technical precision, regardless of absolute value)
→ Obligation: Note offset correction in report

Scenario B (Unstable): Delta > 0.5°C?
→ Rating based on deviation from target (93°C)

• ±0.5°C → 2.0 points
• ±1.5°C → 1.3 points
• >1.5°C → 0.0 points

Cluster B: Intra-shot stability (max. 2.7 points)

What happens during the 25 seconds of extraction?

Time-to-temperature (TTT):
By second 10 at the latest, the target corridor (average ±2°C) must be reached.
If later ("Late riser"): Automatically max. 0.7 points

Curve shape:

• Flatline: Horizontal (Δ < 0.5°C) → 2.7 points (Technical ideal)
• Drop: Slight decline (< 2.0°C) → 2.0 points (Sensorily good – prevents bitterness)
• Rise: Slight increase (< 2.0°C) → 1.3 points (Sensorily critical – aggressive extraction)
• Drift/Unstable: Change > 2.0°C → 0.7 points
• Chaos: → 0.0 points

Cluster C: WBC performance (max. 2.0 points)

Does the machine hold up even at the garden party?

We fire 14 espressos in rapid succession and steam milk in parallel.

Rating:

• Professional level: Window < 1°C (max-min) → 2.0 points
• Household level: Window < 3°C (max-min) → 0.7 points
• Failure: Window > 3°C → 0.0 points

Heat-up time rating grid (rated separately as its own category)

The evaluation goes far beyond the first visual impression. We conduct systematic testing.

The categories (max. 10 points)

A. Surface quality

• Visual inspection of all housing parts (paint, polish, coating)
• Check for uniform gap spacing (< 1mm = excellent, > 1.5mm = problematic)
• Tactile inspection of all edges for sharpness or burrs
• Fit of attachments (drip tray, tank cap, cup rail)

B. Material & haptics

• Identification of primary materials used (metal, plastic, wood)
• Assessment of material thickness (e.g., sheet metal thickness >1.5mm = high quality)
• Tactile inspection of control elements (pressure point, firmness, play)

C. Construction & stability

• Assessment of housing torsional stiffness
• Check of stability
• Quality of visible fasteners and mechanical connections

D. Interior quality (if visible)

• Cleanliness of cable/hose routing as indicator of manufacturing care

Note on durability

We currently do not test the actual durability of an espresso machine. We simply cannot do that seriously for multiple machines. We therefore refer to reports from people who have purchased an espresso machine. Internet sentiment provides a good overview of whether an espresso machine is error-prone.

If we do hear from our own use or from our home barista course participants about repeated problems with test machines, we mention that in our test reports.

Noise level is a decisive comfort factor, especially in open-plan kitchen living spaces. We don't rely on subjective hearing but measure sound pressure level under standardized conditions.

Measurement scenario:

• Distance: 20 cm at brew group height
• Environment: Film studio (controlled acoustic conditions)
• Measured is the maximum value during an espresso shot under load (against blind basket or coffee puck at 9 bar)

Rating grid (0-10 points)

Subjective correction: In extreme cases, the score can be adjusted by max. 1 point if the nature of the sound is particularly disturbing (high-frequency whistling, rattling) or particularly pleasant (sonorous, deep humming). This must be explicitly justified in the test report.

We don't rate the amount of plastic waste, but the quality of the tools. A perfectly fitting tamper is more valuable than five poorly made sieves.

Scoring system by point collection (max. 10 points)

Basic equipment (mandatory elements)

• 2-spout basket included: +1 pt
• At least 2 sieves (different sizes): +1 pt
• Tamper included: +1 pt
• Blind basket for backflushing: +0.5 pt

Quality of core components

• Tamper with perfect fit (<1mm play in basket): +2 pts
• Tamper with good fit (1-2mm play): +1 pt
• Basket made of stainless steel: +0.5 pt
• Sieves burr-free, high-quality stamping: +1.5 pts
• Sieves acceptable: +0.5 pt
• Basket high quality (metal, good haptics): +1 pt

Additional accessories

• 1-spout basket: +0.5 pt
• Bottomless basket: +1 pt
• Milk pitcher (only if high quality): +0.5 pt
• Cleaning product/cleaning brush: +0.5 pt
• Water stick filter: +0.5 pt
• Cups: +0.5 pt
• Useful extras (leveler, WDT tool, apron, etc.): +0.5 pts per item (max. +1 pt total)

Penalty

• Tamper too small (>2mm play): -1 pt
• Burring on sieves: -0.5 pt
• Non-functional cheap accessories: -0.5 pt

Maximum: 10 points (capped if exceeded)

Rates the machine's continuous operation capability ("stress test"). For most home baristas less relevant, but important if you regularly entertain guests.

Point distribution (max. 10 points)

A. Temperature stability under continuous load (0-4 points)

Result from the WBC protocol (Chapter 6):

• 4 pts: WBC protocol passed with window <1°C
• 3 pts: WBC protocol passed with window <3°C
• 2 pts: Slight instability, but still usable
• 1 pt: Clear performance decline
• 0 pts: Failure under load

B. Steam mass potential (0-3 points)

Test: 5 milk steaming operations (300ml each) consecutively at 90-second intervals

• 3 pts: All 5 runs without performance loss, consistent steam power
• 2 pts: Slight performance decline from run 4-5 onward, but still acceptable
• 1 pt: Clear performance decline from run 3 onward
• 0 pts: Machine fails to complete 5 runs

C. Capacities (0-3 points)

• +1.5 pts: Permanent water connection
• +1 pt: Drip tray with drain
• +0.5 pts: Large cup shelf with pre-warming function (>6 cups)

The coffee machine energy score

Energy efficiency in modern espresso machines is no longer a niche topic but a central quality characteristic. To create fair comparability between a small thermoblock and a large dual boiler, we have developed a differentiated rating system. Further explanations of the measurement system available here. Our power protocol is now being used by various espresso machine manufacturers in development.

Our measurement setup

Measuring device: Christ CLM1000 Professional (Plus)

• Resolution: 0.1 Wh (0.0001 kWh)
• Logging: Second-by-second readings as CSV file

Starting conditions:

• Machine at rest for at least 6 hours
• Water tank filled to max with room-temperature water (20-24°C)
• Cold start: Machine unplugged, then plugged in and switched on

The complete measurement protocol

We conduct comprehensive measurement protocol on each machine to cover all usage scenarios:

The rating principle: "Espresso first"

Although we conduct the full protocol, we rate the machine based on the typical daily scenario:

Here, the principle of maximum efficiency applies: If a machine (e.g., a dual boiler) has the function to shut down the energy-hungry steam boiler, we rate it in this economical "espresso mode." We want to reward technological progress and energy-saving features.

Transparency: Additional measured values in our tests

In our detailed test reports, we additionally state:

For classic two-circuit or machines without steam boiler shut-off function, the espresso value is identical to the cappuccino value (minus pure steam shot) – which accordingly reflects in a lower score.

Rating scale (1-10 points)

The scale is based on current technology standards and uses the value "heat-up + 1 double espresso":

The fair value formula

The assessment of price-to-performance ratio (PtP) is not subjective but based on a mathematical "Fair Value" model. This puts the objectively measured overall quality of the machine in relation to its current market price.

Why a formula?

A 3,000-euro machine doesn't have to be three times as good as one for 1,000 euros to have good PtP. This is due to the economic law of diminishing marginal returns: Every additional euro invested brings less additional quality. A perfect 10/10 machine for 5,000 euros can have worse PtP than a solid 7/10 machine for 800 euros.

The calculation model

Step 1: Determine quality basis (0-100 scale)

We take the weighted average of all technical and haptic performance categories (chapters 1-10, WITHOUT the PtP category itself) and scale it to 0-100:

Step 2: Calculate raw value (square root correction)

To avoid penalizing expensive high-end machines linearly, the price is "softened" via square root:

Example:

• Machine A: Quality basis 80, price 1,600 EUR
  → Raw value = (80 / √1600) × 10 = (80 / 40) × 10 = 20.0
• Machine B: Quality basis 70, price 900 EUR
  → Raw value = (70 / √900) × 10 = (70 / 30) × 10 = 23.3

→ Machine B has better PtP despite lower quality!

Step 3: Scale to 0-10 rating (Fair Value 2025)

The raw value is converted to a score based on current market data:

The anchor (score 5.0): A raw value of 15.0 defines the current market standard for fair pricing (e.g., solid mid-range dual boiler).

The slope: A change in raw value of 3.0 points results in a change in score of 1.0.

Formula:

Limit: Min. 0 points, max. 10 points

Interpretation

• Score > 8 Price bargain. Significantly more performance for the price
• Score 5 – 6.5 Fair value. Price matches the performance offered
• Score < 5 Enthusiast premium. You pay a disproportionate markup for nuances, design, or brand

How to interpret our test results

1. The overall score is just the beginning

A 7.5 means "very good" – but maybe the machine is weak in categories unimportant to you and excellent in the ones that matter. Always look at the individual scores!

2. Compare only within similar price ranges

A machine for 800 EUR with score 7.5 is a different beast than one for 2,500 EUR with score 7.5. Use the price-to-performance score for fair comparisons.

3. Identify your "must-haves"

Ask yourself:

• Do I mainly drink espresso or milk drinks? (→ Espresso potential vs. steaming quality)
• Do I use the machine daily or weekends only? (→ Heat-up time, usability)
• Is sustainability important to me? (→ Power consumption)
• Do I often have guests? (→ Volume potential)

4. Use the individual ratings as a checklist

Each category has sub-points. If you know, for example, that smart profiling matters to you, look directly at chapter 1 (Espresso potential) to see if the machine offers it.

5. Wait for individual weighting (coming soon)

Transparency and continuous improvement

This test protocol is a living document. We adapt it continuously based on your feedback and our experience. Version 2.3 is already a significant evolution from our first tests.

What we don't fully test yet

The exact point distribution in the steaming quality and workmanship categories will be provided once we've gathered sufficient comparison data. The categories and evaluation criteria are, however, already defined.

What you can do

• Give us feedback
• Tell us which aspects matter most to you
• Share your experiences with tested machines

Our promise

We stay objective, measurable, and transparent. Every machine is tested by the same protocol. Every machine is bought by us or, in rare cases with very expensive espresso machines, rented by us with a rental fee.

We take our time. Good testing takes time.

Closing remarks

A good test protocol is like a good recipe: It standardizes the process but leaves room for nuance. We hope that with this guide, you not only understand our tests better but also make an informed purchasing decision yourself.

In the end, only one thing matters: The espresso in your cup. All measurements, scores, and formulas are just a means to an end – to help you find the machine that fits your life, your expectations, and your budget.

Good luck with your machine search!

Andrea Perin, Michel Indelicato, Benjamin Hohlmann