Radar Evaluation

This module introduces several evaluators for performance indicators in radar detection. Refer to the PyMonte documentation for a detailed introduction to the concept of Evaluators.

classDiagram Serializable <|-- BitErrorEvaluator Serializable <|-- BlockErrorEvaluator EvaluationTemplate <|-- FrameErrorEvaluation CommunicationEvaluator <|-- BitErrorEvaluator ArtifactTemplate <|-- BlockErrorArtifact Serializable <|-- FrameErrorEvaluator ArtifactTemplate <|-- ThroughputArtifact ArtifactTemplate <|-- FrameErrorArtifact ArtifactTemplate <|-- BitErrorArtifact CommunicationEvaluator <|-- BlockErrorEvaluator FrameErrorEvaluator <|-- ThroughputEvaluator Evaluator <|-- CommunicationEvaluator CommunicationEvaluator <|-- FrameErrorEvaluator EvaluationTemplate <|-- ThroughputEvaluation EvaluationTemplate <|-- BlockErrorEvaluation EvaluationTemplate <|-- BitErrorEvaluation Serializable <|-- ThroughputEvaluator

The implemented RadarEvaluator all inherit from the identically named common base which gets initialized by selecting one Modem, whose performance will be evaluated and one RadarChannel instance, containing the ground truth. The currently considered performance indicators are

Evaluator

Artifact

Performance Indicator

DetectionProbEvaluator

DetectionProbArtifact

Probability of detecting the target at the right bin

ReceiverOperatingCharacteristic

RocArtifact

Probability of detection versus probability of false alarm

:class`.RootMeanSquareError`

RootMeanSquareArtifact

Root mean square error of point detections

Configuring RadarEvaluators to evaluate the radar detection of Modem instances is rather straightforward:

# Create two separate modem instances
modem = Modem()
channel = RadarChannel()

# Create a radar evaluation as an evaluation example
radar_evaluator = DetectionProbEvaluator(modem, channel)

# Extract evaluation
radar_evaluation = radar_evaluator.evaluate()

# Visualize evaluation
radar_evaluation.plot()
class RadarEvaluator(receiving_radar, radar_channel=None)

Bases: Evaluator, ABC

Base class for evaluating communication processes between two modems.

Parameters
  • receiving_radar (Radar) – Modem detecting radar in case of a target.

  • radar_channel (RadarChannel) – Radar channel containing a desired target.

property receiving_radar: Radar

Radar detector with target present.

Returns

Handle to the receiving radar, when target is present.

Return type

Modem

property radar_channel: RadarChannel

Radar channel

Returns

Handle to the radar channel

Return type

RadarChannel

generate_result(grid, artifacts)

Generates an evaluation result from the artifacts collected over the whole simulation grid.

Parameters
  • grid (List[GridDimension]) – The Simulation grid.

  • artifacts (np.ndarray) – Numpy object array whose dimensions represent grid dimensions.

Return type

EvaluationResult

Returns

The evaluation result.

class DetectionProbArtifact(artifact)

Bases: ArtifactTemplate[bool]

Artifact of a detection probability evaluation for a radar detector.

Parameters

artifact (AT) – Artifact value.

to_scalar()

Scalar representation of this artifact’s content.

Used to evaluate premature stopping criteria for the underlying evaluation.

Returns

Scalar floating-point representation. None if a conversion to scalar is impossible.

Return type

Optional[float]

class DetectionProbabilityEvaluation(evaluation)

Bases: EvaluationTemplate[bool]

artifact()

Generate an artifact from this evaluation.

Returns: The evaluation artifact.

Return type

DetectionProbArtifact

evaluation: ET
class DetectionProbEvaluator(receiving_radar)

Bases: RadarEvaluator, Serializable

Evaluate detection probability at a radar detector, considering any bin, i.e., detection is considered if any bin in the radar cube is above the threshold

Parameters

receiving_radar (Radar) – Radar detector

yaml_tag: Optional[str] = 'DetectionProbEvaluator'

YAML serialization tag

property abbreviation: str

Short string representation of this evaluator.

Used as a label for console output and plot axes annotations.

Returns

String representation

Return type

str

property title: str

Long string representation of this evaluator.

Used as plot title.

Returns

String representation

Return type

str

generate_result(grid, artifacts)

Generates an evaluation result from the artifacts collected over the whole simulation grid.

Parameters
  • grid (List[GridDimension]) – The Simulation grid.

  • artifacts (np.ndarray) – Numpy object array whose dimensions represent grid dimensions.

Return type

ScalarEvaluationResult

Returns

The evaluation result.

evaluate()

Evaluate the state of an investigated object.

Implements the process of extracting an arbitrary performance indicator, represented by the returned Artifact \(X_m\). :returns: Artifact \(X_m\) resulting from the evaluation. :rtype: Artifact

class RocArtifact(h0_value, h1_value)

Bases: Artifact

Artifact of receiver operating characteristics (ROC) evaluation

Parameters
  • h0_value (float) – Measured value for null-hypothesis (H0), i.e., noise only

  • h1_value (float) – Measured value for alternative hypothesis (H1)

property h0_value: float
Return type

float

property h1_value: float
Return type

float

class RocEvaluation(cube_h0, cube_h1)

Bases: Evaluation

Evaluation of receiver operating characteristics (ROC)

data_h0: ndarray
data_h1: ndarray
artifact()

Generate an artifact from this evaluation.

Returns: The evaluation artifact.

Return type

RocArtifact

class RocEvaluationResult(evaluator, grid, detection_probabilities, false_alarm_probabilities)

Bases: EvaluationResult

Final result of an receive operating characteristcs evaluation.

plot()

Render a visual representation of the evaluation result.

Generated by matplotlib.

Return type

Figure

Returns

A handle to the generated matplotlib figure.

to_array()

Convert the evaluation result raw data to an array representation.

Used to store the results in arbitrary binary file formats after simulation execution.

Return type

ndarray

Returns

The array result representation.

class ReceiverOperatingCharacteristic(receiving_radar, receiving_radar_null_hypothesis, radar_channel=None, num_thresholds=101)

Bases: RadarEvaluator, Serializable

Evaluate the receiver operating characteristics for a radar operator.

Parameters
  • receiving_radar (Radar) – Modem detecting radar in case of a target.

  • receiving_radar_null_hypothesis (Radar) – Radar receiver containing only noise.

  • radar_channel (RadarChannel, Optional) – Radar channel containing a desired target. If a radar channel is given, then the ROC is calculated for the bin that contains the target, or else a detection is performed if the output of any bin is above the threshold.

  • num_thresholds (int, Optional) – Number of different thresholds to be considered in ROC curve

yaml_tag: Optional[str] = 'ROC'

YAML serialization tag.

evaluate()

Evaluate the state of an investigated object.

Implements the process of extracting an arbitrary performance indicator, represented by the returned Artifact \(X_m\). :returns: Artifact \(X_m\) resulting from the evaluation. :rtype: Artifact

property receiving_radar_null_hypothesis: Radar

Radar detector with only noise

Returns

Handle to the receiving modem, with only noise at receiver.

Return type

Modem

property abbreviation: str

Short string representation of this evaluator.

Used as a label for console output and plot axes annotations.

Returns

String representation

Return type

str

property title: str

Long string representation of this evaluator.

Used as plot title.

Returns

String representation

Return type

str

generate_result(grid, artifacts)

Generates an evaluation result from the artifacts collected over the whole simulation grid.

Parameters
  • grid (List[GridDimension]) – The Simulation grid.

  • artifacts (np.ndarray) – Numpy object array whose dimensions represent grid dimensions.

Return type

EvaluationResult

Returns

The evaluation result.

class RootMeanSquareArtifact(num_errors, cummulation)

Bases: Artifact

Artifact of a root mean square evaluation

Parameters
  • num_errors (int) – Number of errros.

  • cummulation (float) – Sum of squared errors distances.

num_errors: int
cummulation: float
to_scalar()

Scalar representation of this artifact’s content.

Used to evaluate premature stopping criteria for the underlying evaluation.

Returns

Scalar floating-point representation. None if a conversion to scalar is impossible.

Return type

Optional[float]

class RootMeanSquareEvaluation(pcl, ground_truth)

Bases: Evaluation

Result of a single root mean squre evaluation.

artifact()

Generate an artifact from this evaluation.

Returns: The evaluation artifact.

Return type

RootMeanSquareArtifact

class RootMeanSquareErrorResult(grid, scalar_results, evaluator, plot_surface=True)

Bases: ProcessedScalarEvaluationResult

Result of a root mean square error evaluation.

plot_surface: bool
class RootMeanSquareError(receiving_radar, radar_channel=None)

Bases: RadarEvaluator

Root mean square estimation error of point detections.

Parameters
  • receiving_radar (Radar) – Modem detecting radar in case of a target.

  • radar_channel (RadarChannel) – Radar channel containing a desired target.

evaluate()

Evaluate the state of an investigated object.

Implements the process of extracting an arbitrary performance indicator, represented by the returned Artifact \(X_m\). :returns: Artifact \(X_m\) resulting from the evaluation. :rtype: Artifact

property title: str

Long string representation of this evaluator.

Used as plot title.

Returns

String representation

Return type

str

property abbreviation: str

Short string representation of this evaluator.

Used as a label for console output and plot axes annotations.

Returns

String representation

Return type

str

generate_result(grid, artifacts)

Generates an evaluation result from the artifacts collected over the whole simulation grid.

Parameters
  • grid (List[GridDimension]) – The Simulation grid.

  • artifacts (np.ndarray) – Numpy object array whose dimensions represent grid dimensions.

Return type

RootMeanSquareErrorResult

Returns

The evaluation result.