Radar Evaluation

This module introduces several evaluators for performance indicators in radar detection. Refer to the PyMonte documentation for a detailed introduction to the concept of Evaluators.

The implemented RadarEvaluator all inherit from the identically named common base which gets initialized by selecting one Modem, whose performance will be evaluated and one RadarChannel instance, containing the ground truth. The currently considered performance indicators are

Evaluator	Artifact	Performance Indicator
`DetectionProbEvaluator`	`DetectionProbArtifact`	Probability of detecting the target at the right bin
`ReceiverOperatingCharacteristic`	`RocArtifact`	Probability of detection versus probability of false alarm
:class`.RootMeanSquareError`	`RootMeanSquareArtifact`	Root mean square error of point detections

Configuring RadarEvaluators to evaluate the radar detection of Modem instances is rather straightforward:

# Create two separate modem instances
modem = Modem()
channel = RadarChannel()

# Create a radar evaluation as an evaluation example
radar_evaluator = DetectionProbEvaluator(modem, channel)

# Extract evaluation
radar_evaluation = radar_evaluator.evaluate()

# Visualize evaluation
radar_evaluation.plot()

class RadarEvaluator(receiving_radar, radar_channel=None)

Bases: Evaluator, ABC

Base class for evaluating communication processes between two modems.

Parameters

receiving_radar (Radar) – Modem detecting radar in case of a target.
radar_channel (RadarChannel) – Radar channel containing a desired target.

property receiving_radar: Radar

Radar detector with target present.

Returns: Handle to the receiving radar, when target is present.
Return type: Modem

property radar_channel: RadarChannel

Radar channel

Returns: Handle to the radar channel
Return type: RadarChannel

generate_result(grid, artifacts)

Generates an evaluation result from the artifacts collected over the whole simulation grid.

Parameters

grid (List[GridDimension]) – The Simulation grid.
artifacts (np.ndarray) – Numpy object array whose dimensions represent grid dimensions.

Return type

EvaluationResult

Returns

The evaluation result.

class DetectionProbArtifact(artifact)

Bases: ArtifactTemplate[bool]

Artifact of a detection probability evaluation for a radar detector.

Parameters: artifact (AT) – Artifact value.

to_scalar()

Scalar representation of this artifact’s content.

Used to evaluate premature stopping criteria for the underlying evaluation.

Returns: Scalar floating-point representation. None if a conversion to scalar is impossible.
Return type: Optional[float]

class DetectionProbabilityEvaluation(evaluation)

Bases: EvaluationTemplate[bool]

artifact()

Generate an artifact from this evaluation.

Returns: The evaluation artifact.

Return type: DetectionProbArtifact

evaluation: ET

class DetectionProbEvaluator(receiving_radar)

Bases: RadarEvaluator, Serializable

Evaluate detection probability at a radar detector, considering any bin, i.e., detection is considered if any bin in the radar cube is above the threshold

Parameters: receiving_radar (Radar) – Radar detector

yaml_tag: Optional[str] = 'DetectionProbEvaluator': YAML serialization tag

property abbreviation: str

Short string representation of this evaluator.

Used as a label for console output and plot axes annotations.

Returns: String representation
Return type: str

property title: str

Long string representation of this evaluator.

Used as plot title.

Returns: String representation
Return type: str

generate_result(grid, artifacts)

Generates an evaluation result from the artifacts collected over the whole simulation grid.

Parameters

grid (List[GridDimension]) – The Simulation grid.
artifacts (np.ndarray) – Numpy object array whose dimensions represent grid dimensions.

Return type

ScalarEvaluationResult

Returns

The evaluation result.

evaluate()

Evaluate the state of an investigated object.

Implements the process of extracting an arbitrary performance indicator, represented by the returned Artifact \(X_m\). :returns: Artifact \(X_m\) resulting from the evaluation. :rtype: Artifact

class RocArtifact(h0_value, h1_value)

Bases: Artifact

Artifact of receiver operating characteristics (ROC) evaluation

Parameters

h0_value (float) – Measured value for null-hypothesis (H0), i.e., noise only
h1_value (float) – Measured value for alternative hypothesis (H1)

property h0_value: float

Return type: float

property h1_value: float

Return type: float

class RocEvaluation(cube_h0, cube_h1)

Bases: Evaluation

Evaluation of receiver operating characteristics (ROC)

data_h0: ndarray

data_h1: ndarray

artifact()

Generate an artifact from this evaluation.

Returns: The evaluation artifact.

Return type: RocArtifact

class RocEvaluationResult(evaluator, grid, detection_probabilities, false_alarm_probabilities)

Bases: EvaluationResult

Final result of an receive operating characteristcs evaluation.

plot()

Render a visual representation of the evaluation result.

Generated by matplotlib.

Return type: Figure
Returns: A handle to the generated matplotlib figure.

to_array()

Convert the evaluation result raw data to an array representation.

Used to store the results in arbitrary binary file formats after simulation execution.

Return type: ndarray
Returns: The array result representation.

class ReceiverOperatingCharacteristic(receiving_radar, receiving_radar_null_hypothesis, radar_channel=None, num_thresholds=101)

Bases: RadarEvaluator, Serializable

Evaluate the receiver operating characteristics for a radar operator.

Parameters

receiving_radar (Radar) – Modem detecting radar in case of a target.
receiving_radar_null_hypothesis (Radar) – Radar receiver containing only noise.
radar_channel (RadarChannel, Optional) – Radar channel containing a desired target. If a radar channel is given, then the ROC is calculated for the bin that contains the target, or else a detection is performed if the output of any bin is above the threshold.
num_thresholds (int, Optional) – Number of different thresholds to be considered in ROC curve

yaml_tag: Optional[str] = 'ROC': YAML serialization tag.

evaluate()

Evaluate the state of an investigated object.

Implements the process of extracting an arbitrary performance indicator, represented by the returned Artifact \(X_m\). :returns: Artifact \(X_m\) resulting from the evaluation. :rtype: Artifact

property receiving_radar_null_hypothesis: Radar

Radar detector with only noise

Returns: Handle to the receiving modem, with only noise at receiver.
Return type: Modem

property abbreviation: str

Short string representation of this evaluator.

Used as a label for console output and plot axes annotations.

Returns: String representation
Return type: str

property title: str

Long string representation of this evaluator.

Used as plot title.

Returns: String representation
Return type: str

generate_result(grid, artifacts)

Generates an evaluation result from the artifacts collected over the whole simulation grid.

Parameters

grid (List[GridDimension]) – The Simulation grid.
artifacts (np.ndarray) – Numpy object array whose dimensions represent grid dimensions.

Return type

EvaluationResult

Returns

The evaluation result.

class RootMeanSquareArtifact(num_errors, cummulation)

Bases: Artifact

Artifact of a root mean square evaluation

Parameters

num_errors (int) – Number of errros.
cummulation (float) – Sum of squared errors distances.

num_errors: int

cummulation: float

to_scalar()

Scalar representation of this artifact’s content.

Used to evaluate premature stopping criteria for the underlying evaluation.

Returns: Scalar floating-point representation. None if a conversion to scalar is impossible.
Return type: Optional[float]

class RootMeanSquareEvaluation(pcl, ground_truth)

Bases: Evaluation

Result of a single root mean squre evaluation.

artifact()

Generate an artifact from this evaluation.

Returns: The evaluation artifact.

Return type: RootMeanSquareArtifact

class RootMeanSquareErrorResult(grid, scalar_results, evaluator, plot_surface=True)

Bases: ProcessedScalarEvaluationResult

Result of a root mean square error evaluation.

plot_surface: bool

class RootMeanSquareError(receiving_radar, radar_channel=None)

Bases: RadarEvaluator

Root mean square estimation error of point detections.

Parameters

receiving_radar (Radar) – Modem detecting radar in case of a target.
radar_channel (RadarChannel) – Radar channel containing a desired target.

evaluate()

Evaluate the state of an investigated object.

Implements the process of extracting an arbitrary performance indicator, represented by the returned Artifact \(X_m\). :returns: Artifact \(X_m\) resulting from the evaluation. :rtype: Artifact

property title: str

Long string representation of this evaluator.

Used as plot title.

Returns: String representation
Return type: str

property abbreviation: str

Short string representation of this evaluator.

Used as a label for console output and plot axes annotations.

Returns: String representation
Return type: str

generate_result(grid, artifacts)

Generates an evaluation result from the artifacts collected over the whole simulation grid.

Parameters

grid (List[GridDimension]) – The Simulation grid.
artifacts (np.ndarray) – Numpy object array whose dimensions represent grid dimensions.

Return type

RootMeanSquareErrorResult

Returns

The evaluation result.