Running SCPI instrument tests in CI without hardware
How to test end-of-line test station code in pytest and CI against simulated SCPI instruments: a supply in current limit, a dead multimeter, a wrong instrument.
Test station software has an awkward property: it can only be tested where the instruments are. The power supply and the multimeter are on the production line or on a colleague’s bench, and they are busy. So the code that decides whether a board passes or fails is usually tested by running it once on a good board and calling it done.
That leaves the interesting cases untested: the rail that sits in current limit, the multimeter that stops answering in the middle of a shift, the instrument that was swapped for another model. This article shows how to test exactly those cases with pytest, on every commit, without a single instrument.
The station
A typical end-of-line test for a board with a 12 V and a 3.3 V rail: a two-channel power supply feeds the board, a relay multiplexer routes one rail at a time to a multimeter, and the code compares each reading with its limits. The station code talks to the instruments through PyVISA:
# station.py
"""End-of-line test for a board with a 12 V and a 3.3 V rail."""
RAILS = {101: ("12V", 12.0), 102: ("3V3", 3.3)} # mux channel: (name, nominal)
TOLERANCE = 0.05 # +/- 5 %
class StationError(Exception):
pass
def check_errors(inst) -> None:
"""Read the error queue empty; any entry means a command went wrong."""
errors = []
while not (reply := inst.query("SYST:ERR?")).startswith(("+0", "0")):
errors.append(reply)
if errors:
raise StationError(f"instrument reported {errors}")
def check_identity(inst, model: str) -> None:
idn = inst.query("*IDN?")
if model not in idn:
raise StationError(f"expected a {model}, found {idn!r}")
def power_up(psu) -> None:
psu.query("SOUR1:VOLT 12;CURR 1;:SOUR2:VOLT 3.3;CURR 1;:OUTP1 ON;:OUTP2 ON;*OPC?")
check_errors(psu)
def measure_rail(dmm, mux, channel: int) -> float:
# *OPC? waits until the relay has switched: the DMM is another connection,
# so a plain write could still be in flight when it measures.
mux.query(f"ROUT:OPEN:ALL;:ROUT:CLOS (@{channel});*OPC?")
return float(dmm.query("MEAS:VOLT:DC?"))
def run(psu, dmm, mux) -> dict[str, str]:
check_identity(dmm, "DMM")
power_up(psu)
result = {}
for channel, (name, nominal) in RAILS.items():
volts = measure_rail(dmm, mux, channel)
ok = abs(volts - nominal) <= nominal * TOLERANCE
result[name] = f"{'PASS' if ok else 'FAIL'} {volts:.3f} V"
return result
Nothing special, and that is the point: this is the kind of code that runs on test stations everywhere, and most of it has only ever met a good board.
Simulated instruments that behave
scpi-sim is an open-source Python
package I wrote for this. It runs simulated SCPI instruments (power supply,
multimeter, relay multiplexer, oscilloscope) as real network servers on a raw
socket, and on request over VXI-11 and HiSLIP. PyVISA, C#, LabVIEW or nc
talk to them like to real LAN instruments. Profiles make them answer like
fifteen common models from Rohde & Schwarz, Keysight, Keithley, Siglent,
Rigol and Tektronix.
What matters for testing is that they behave, not just answer:
- the supply goes from constant voltage into current limit when its load draws more than the limit allows, and its output voltage drops;
- instruments on a bench are wired together, so the multimeter measures what the supply actually delivers into the load;
- they are strict where real instruments are strict: a malformed command ends up in the error queue, not silently accepted;
- faults can be injected at runtime: no answer, a late answer, a dropped connection, a wrong identity, drifting readings.

You can try it in your browser without installing anything.
The tests
Installing scpi-sim registers a pytest plugin. scpi_bench_factory starts
a bench from a dict or a YAML file, on free ports, for each test, and stops it
afterwards. The bench below is the station above: two rails as resistive
loads, wired to the supply and through the multiplexer to the multimeter.
# test_station.py
import pytest
import pyvisa
import station
BENCH = {
"instruments": {
"psu": {"type": "psu", "port": 0, "channels": 2},
"dmm": {"type": "dmm", "port": 0},
"mux": {"type": "mux", "port": 0, "channels": 16},
},
"loads": {
"rail_12v": {"type": "resistor", "ohms": 24}, # 0.5 A at 12 V
"rail_3v3": {"type": "resistor", "ohms": 33}, # 0.1 A at 3.3 V
},
"wiring": [
"psu.ch1 -> rail_12v", "psu.ch2 -> rail_3v3",
"mux.101 -> rail_12v", "mux.102 -> rail_3v3",
"mux.common -> dmm.input",
],
}
@pytest.fixture
def bench(scpi_bench_factory):
return scpi_bench_factory(BENCH)
@pytest.fixture
def instruments(bench):
rm = pyvisa.ResourceManager("@py")
insts = [rm.open_resource(bench.resource(name), read_termination="\n",
write_termination="\n", timeout=500)
for name in ("psu", "dmm", "mux")]
yield insts
for inst in insts:
inst.close()
rm.close()
def test_good_board_passes(instruments):
result = station.run(*instruments)
assert result["12V"].startswith("PASS")
assert result["3V3"].startswith("PASS")
def test_rail_in_current_limit_fails(instruments):
psu, dmm, mux = instruments
station.power_up(psu)
psu.query("SOUR2:CURR 0.05;*OPC?") # the 3.3 V rail needs 0.1 A
assert psu.query("OUTP2:MODE?") == "CC"
assert station.measure_rail(dmm, mux, 102) < 3.3 * 0.95
def test_wrong_instrument_is_refused(bench, instruments):
bench.dmm.inject("idn", value="ACME,PSU-9000,1234,1.0")
with pytest.raises(station.StationError, match="expected a DMM"):
station.run(*instruments)
def test_dead_multimeter_times_out(bench, instruments):
psu, dmm, mux = instruments
station.power_up(psu)
bench.dmm.inject("no_response")
with pytest.raises(pyvisa.errors.VisaIOError):
station.measure_rail(dmm, mux, 101)
def test_typo_in_a_command_is_caught(instruments):
psu = instruments[0]
psu.write("SOUR1:VOLT 12;SOUR1:CURR 1") # path rule: needs ":" before SOUR1
with pytest.raises(station.StationError, match="-113"):
station.check_errors(psu)
$ pytest -q
..... [100%]
5 passed in 0.66s
The good board reads {'12V': 'PASS 12.000 V', '3V3': 'PASS 3.300 V'}. The
other four tests are the ones that matter.
Three things the tests catch
A rail in current limit. Lower the current limit below what the 3.3 V
rail draws, and the supply switches to constant current (OUTP2:MODE?
answers CC); the voltage on the rail drops, and the station must fail the
board. With a static simulator that only echoes a configured value, this test
would pass for the wrong reason.
A dead or wrong instrument. no_response makes the multimeter swallow its
next answer, so the station must see a timeout, not hang and not record a
stale value. idn makes the instrument claim to be something else, which is
what happens when somebody swaps a unit on the line. Both take one line in the
test and would otherwise only show up in production.
The SCPI path rule. This one surprises people. In a compound command,
everything after a ; continues at the level of the previous command. So
SOUR1:VOLT 12;SOUR1:CURR 1
is read as SOUR1:VOLT 12 followed by SOUR1:SOUR1:CURR 1, which does not
exist. A real instrument answers with -113,"Undefined header" in its error
queue, keeps the voltage and ignores the current limit, and carries on. The
correct form is SOUR1:VOLT 12;:SOUR1:CURR 1 (or SOUR1:VOLT 12;CURR 1).
The simulator is just as strict, so check_errors after configuration
finds the mistake before it meets the hardware:
>>> psu.write("SOUR1:VOLT 12;SOUR1:CURR 1")
>>> psu.query("SYST:ERR?")
'-113,"Undefined header"'
>>> psu.query("SYST:ERR?")
'0,"No error"'
Two habits from this example are worth keeping on any station, simulated or
not: read the error queue after configuring an instrument, and finish
commands with *OPC? before measuring on another connection. Each
instrument is its own connection; without *OPC?, the multimeter can measure
before the relay has switched.
In CI
The pytest plugin starts the instruments itself, so CI only needs the
packages. PyVISA with the pure-Python backend (pyvisa-py) needs no
NI-VISA installation:
# .github/workflows/test.yml
name: test
on: [push, pull_request]
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
with:
python-version: "3.12"
- run: pip install scpi-sim pyvisa pyvisa-py pytest
- run: pytest -q
Station code written in C# or LabVIEW can use the Docker image instead,
ghcr.io/steitec/scpi-sim, which runs a bench like the one above and serves
raw sockets, VXI-11 and HiSLIP; faults are injected over its control port.
The README has the details.
Limits
The instruments behave plausibly, not exactly like one particular unit: noise, settling times and the quirks of a given firmware are simplified, and the profiles follow the manuals rather than tests against every real model. VXI-11 and HiSLIP are tested with PyVISA-py, not yet with NI-VISA or Keysight VISA. None of this replaces the first run on the real station. What it replaces is the hope that the station code copes with a board in current limit, a dead multimeter or a typo in a command string. That you can now test on every commit.