Reiver Operating Characteristic

class ReceiverOperatingCharacteristic(receiving_radar, transmitting_device, receiving_device, radar_channel, num_thresholds=101)[source]

Bases: RadarEvaluator, Serializable

Evaluate the receiver operating characteristics for a radar operator.

The receiver operating characteristics (ROC) curve is a graphical plot that illustrates the performance of a detector by visualizing the probability of false alarm versus the probability of detection for a given parameterization.

A minimal example within the context of a Simulation evaluating the probability of detection for a single radar target illuminated by an FMCW radar would be:

 1from hermespy.radar import Radar, FMCW, ReceiverOperatingCharacteristic
 2from hermespy.simulation import Simulation
 3from hermespy.channel import SingleTargetRadarChannel
 4
 5# Create a new simulated scenario featuring a single device
 6simulation = Simulation()
 7device = simulation.new_device(carrier_frequency=60e9)
 8
 9# Configure the device to transmit and reveive radar waveforms
10radar = Radar(waveform=FMCW())
11device.add_dsp(radar)
12
13# Create a new radar channel with a single illuminated target
14target = SingleTargetRadarChannel(1, 1., attenuate=True)
15simulation.scenario.set_channel(device, device, target)
16
17# Create a new detection probability evaluator
18simulation.add_evaluator(ReceiverOperatingCharacteristic(radar, device, device, target))
19
20# Run the simulation
21result = simulation.run()
Parameters:

  • radar – Radar under test.

  • radar_channel (RadarChannelBase) – Radar channel containing a desired target.

  • num_thresholds – Number of different thresholds to be considered in ROC curve

classmethod FromFile(file, h0_campaign='h0_measurements', h1_campaign='h1_measurements', num_thresholds=101, backend=SerializationBackend.HDF)[source]

Compute an ROC evaluation result from a savefile.

Parameters:

  • file (str) – Savefile containing the measurements. Either as file system location or h5py File handle.

  • h0_campaign – Campaign identifier of the h0 hypothesis measurements. By default, h0_measurements is assumed.

  • h1_campaign – Campaign identifier of the h1 hypothesis measurements. By default, h1_measurements is assumed.

  • num_thresholds (int) – Number of different thresholds to be considered in ROC curve

  • backend (SerializationBackend) – Serialization backend to be used for the evaluation. By default, HDF is assumed.

Return type:

RocEvaluationResult

Returns: The ROC evaluation result.

static FromScenario(scenario, operator, h0_campaign='h0', h1_campaign='h1', backend=SerializationBackend.HDF, console=None)[source]

Extract an ROC evaluation from an existing scenario.

Parameters:

  • scenario (Scenario) – Scenario from which to extract the ROC from.

  • operator (Radar) – Radar operator to be evaluated.

  • h0_campaign (str) – Campaign identifier of the null hypothesis measurements.

  • h1_campaign (str) – Campaign identifier of the alternative hypothesis measurements.

  • backend (SerializationBackend) – Serialization backend to be used for the evaluation.

  • console (Console | None) – Rich console to be used for progress tracking.

Raises:

ValueError – If the scenario is not in replay mode.

Return type:

RocEvaluationResult

Returns:

The ROC evaluation result.

static FromScenarios(h0_scenario, h1_scenario, num_drops, h0_operator=None, h1_operator=None, num_thresholds=101)[source]

Compute an ROC evaluation result from two scenarios.

Parameters:

  • h0_scenario (Scenario) – Scenario of the null hypothesis.

  • h1_scenario (Scenario) – Scenario of the alternative hypothesis.

  • num_drops (int) – Number of drops to be considered in the evaluation. The more drops, the smoother the estimated ROC curve will be.

  • h0_operator (Radar | None) – Radar operator of the null hypothesis. If not provided, the first radar operator of the null hypothesis scenario will be used.

  • h1_operator (Radar | None) – Radar operator of the alternative hypothesis. If not provided, the first radar operator of the alternative hypothesis scenario will be used.

  • num_thresholds (int) – Number of different thresholds to be considered in ROC curve

Return type:

RocEvaluationResult

Returns: The ROC evaluation result.

Raises:

ValueError –

  • If the number of drops is less than one - If the operators are not registered within the scenarios - If, for any reason, the operated devices cannot be determined

static GenerateResult(grid, artifacts, num_thresholds=101, evaluator=None)[source]

Generate a new receiver operating characteristics evaluation result.

Parameters:

  • grid (Sequence[GridDimension]) – Grid dimensions of the evaluation result.

  • artifacts (ndarray) – Artifacts of the evaluation result.

  • num_thresholds (int) – Number of different thresholds to be considered in ROC curve 101 by default.

  • evaluator (ReceiverOperatingCharacteristic | None) – Evaluator that generated the evaluation result.

Return type:

RocEvaluationResult

Returns: The generated result.

evaluate()[source]

Evaluate the state of an investigated object.

Implements the process of extracting an arbitrary performance indicator, represented by the returned Artifact \(X_m\).

Returns: Artifact \(X_m\) resulting from the evaluation.

Return type:

RocEvaluation

generate_result(grid, artifacts)[source]

Generate a new receiver operating characteristics evaluation result.

Parameters:
Return type:

RocEvaluationResult

Returns: The generated result.

property abbreviation: str

Short string representation of this evaluator.

Used as a label for console output and plot axes annotations.

property title: str

Long string representation of this evaluator.

Used as plot title.

class RocEvaluation(cube_h0, cube_h1)[source]

Bases: Evaluation[PlotVisualization]

Evaluation of receiver operating characteristics (ROC)

Parameters:

  • cube_h0 (RadarCube) – H0 hypothesis radar cube.

  • cube_h1 (RadarCube) – H1 hypothesis radar cube.

artifact()[source]

Generate an artifact from this evaluation.

Returns: The evaluation artifact.

Return type:

RocArtifact

property cube_h0: RadarCube

H0 hypothesis radar cube.

property cube_h1: RadarCube

H1 hypothesis radar cube.

class RocArtifact(h0_value, h1_value)[source]

Bases: Artifact

Artifact of receiver operating characteristics (ROC) evaluation

Parameters:

  • h0_value (float) – Measured value for null-hypothesis (H0), i.e., noise only

  • h1_value (float) – Measured value for alternative hypothesis (H1)

to_scalar()[source]

Scalar representation of this artifact’s content.

Used to evaluate premature stopping criteria for the underlying evaluation.

Return type:

None

Returns:

Scalar floating-point representation. None if a conversion to scalar is impossible.

property h0_value: float
property h1_value: float
class RocEvaluationResult(detection_probabilities, false_alarm_probabilities, grid, evaluator=None)[source]

Bases: EvaluationResult

Final result of an receive operating characteristcs evaluation.

Parameters:

  • detection_probabilities (ndarray) – Detection probabilities for each grid point.

  • false_alarm_probabilities (ndarray) – False alarm probabilities for each grid point.

  • grid (Sequence[GridDimension]) – Grid dimensions of the evaluation result.

  • evaluator (ReceiverOperatingCharacteristic | None) – Evaluator that generated the evaluation result.

to_array()[source]

Convert the evaluation result raw data to an array representation.

Used to store the results in arbitrary binary file formats after simulation execution.

Returns: The array result representation.

Return type:

ndarray