Radar Evaluation
This module introduces several evaluators for performance indicators in radar detection.
Refer to the PyMonte documentation for a detailed introduction to the concept of
Evaluators
.
The implemented RadarEvaluator
all inherit from the identically named common
base which gets initialized by selecting one Modem
, whose performance will be
evaluated and one RadarChannel
instance, containing the ground
truth.
The currently considered performance indicators are
Evaluator |
Artifact |
Performance Indicator |
---|---|---|
Probability of detecting the target at the right bin |
||
Probability of detection versus probability of false alarm |
||
:class`.RootMeanSquareError` |
Root mean square error of point detections |
Configuring RadarEvaluators
to evaluate the radar detection of
Modem
instances is rather straightforward:
# Create two separate modem instances
modem = Modem()
channel = RadarChannel()
# Create a radar evaluation as an evaluation example
radar_evaluator = DetectionProbEvaluator(modem, channel)
# Extract evaluation
radar_evaluation = radar_evaluator.evaluate()
# Visualize evaluation
radar_evaluation.plot()
- class RadarEvaluator(receiving_radar, radar_channel=None)
Bases:
Evaluator
,ABC
Base class for evaluating communication processes between two modems.
- Parameters
receiving_radar (Radar) – Modem detecting radar in case of a target.
radar_channel (RadarChannel) – Radar channel containing a desired target.
- property receiving_radar: Radar
Radar detector with target present.
- Returns
Handle to the receiving radar, when target is present.
- Return type
Modem
- property radar_channel: RadarChannel
Radar channel
- Returns
Handle to the radar channel
- Return type
- generate_result(grid, artifacts)
Generates an evaluation result from the artifacts collected over the whole simulation grid.
- Parameters
grid (List[GridDimension]) – The Simulation grid.
artifacts (np.ndarray) – Numpy object array whose dimensions represent grid dimensions.
- Return type
- Returns
The evaluation result.
- class DetectionProbArtifact(artifact)
Bases:
ArtifactTemplate
[bool
]Artifact of a detection probability evaluation for a radar detector.
- Parameters
artifact (AT) – Artifact value.
- to_scalar()
Scalar representation of this artifact’s content.
Used to evaluate premature stopping criteria for the underlying evaluation.
- Returns
Scalar floating-point representation. None if a conversion to scalar is impossible.
- Return type
Optional[float]
- class DetectionProbabilityEvaluation(evaluation)
Bases:
EvaluationTemplate
[bool
]- artifact()
Generate an artifact from this evaluation.
Returns: The evaluation artifact.
- Return type
- evaluation: ET
- class DetectionProbEvaluator(receiving_radar)
Bases:
RadarEvaluator
,Serializable
Evaluate detection probability at a radar detector, considering any bin, i.e., detection is considered if any bin in the radar cube is above the threshold
- Parameters
receiving_radar (Radar) – Radar detector
- yaml_tag: Optional[str] = 'DetectionProbEvaluator'
YAML serialization tag
- property abbreviation: str
Short string representation of this evaluator.
Used as a label for console output and plot axes annotations.
- Returns
String representation
- Return type
str
- property title: str
Long string representation of this evaluator.
Used as plot title.
- Returns
String representation
- Return type
str
- generate_result(grid, artifacts)
Generates an evaluation result from the artifacts collected over the whole simulation grid.
- Parameters
grid (List[GridDimension]) – The Simulation grid.
artifacts (np.ndarray) – Numpy object array whose dimensions represent grid dimensions.
- Return type
- Returns
The evaluation result.
- class RocArtifact(h0_value, h1_value)
Bases:
Artifact
Artifact of receiver operating characteristics (ROC) evaluation
- Parameters
h0_value (float) – Measured value for null-hypothesis (H0), i.e., noise only
h1_value (float) – Measured value for alternative hypothesis (H1)
- property h0_value: float
- Return type
float
- property h1_value: float
- Return type
float
- class RocEvaluation(cube_h0, cube_h1)
Bases:
Evaluation
Evaluation of receiver operating characteristics (ROC)
- data_h0: ndarray
- data_h1: ndarray
- artifact()
Generate an artifact from this evaluation.
Returns: The evaluation artifact.
- Return type
- class RocEvaluationResult(evaluator, grid, detection_probabilities, false_alarm_probabilities)
Bases:
EvaluationResult
Final result of an receive operating characteristcs evaluation.
- plot()
Render a visual representation of the evaluation result.
Generated by matplotlib.
- Return type
Figure
- Returns
A handle to the generated matplotlib figure.
- to_array()
Convert the evaluation result raw data to an array representation.
Used to store the results in arbitrary binary file formats after simulation execution.
- Return type
ndarray
- Returns
The array result representation.
- class ReceiverOperatingCharacteristic(receiving_radar, receiving_radar_null_hypothesis, radar_channel=None, num_thresholds=101)
Bases:
RadarEvaluator
,Serializable
Evaluate the receiver operating characteristics for a radar operator.
- Parameters
receiving_radar (Radar) – Modem detecting radar in case of a target.
receiving_radar_null_hypothesis (Radar) – Radar receiver containing only noise.
radar_channel (RadarChannel, Optional) – Radar channel containing a desired target. If a radar channel is given, then the ROC is calculated for the bin that contains the target, or else a detection is performed if the output of any bin is above the threshold.
num_thresholds (int, Optional) – Number of different thresholds to be considered in ROC curve
- yaml_tag: Optional[str] = 'ROC'
YAML serialization tag.
- evaluate()
Evaluate the state of an investigated object.
Implements the process of extracting an arbitrary performance indicator, represented by the returned
Artifact
\(X_m\). :returns: Artifact \(X_m\) resulting from the evaluation. :rtype: Artifact
- property receiving_radar_null_hypothesis: Radar
Radar detector with only noise
- Returns
Handle to the receiving modem, with only noise at receiver.
- Return type
Modem
- property abbreviation: str
Short string representation of this evaluator.
Used as a label for console output and plot axes annotations.
- Returns
String representation
- Return type
str
- property title: str
Long string representation of this evaluator.
Used as plot title.
- Returns
String representation
- Return type
str
- generate_result(grid, artifacts)
Generates an evaluation result from the artifacts collected over the whole simulation grid.
- Parameters
grid (List[GridDimension]) – The Simulation grid.
artifacts (np.ndarray) – Numpy object array whose dimensions represent grid dimensions.
- Return type
- Returns
The evaluation result.
- class RootMeanSquareArtifact(num_errors, cummulation)
Bases:
Artifact
Artifact of a root mean square evaluation
- Parameters
num_errors (int) – Number of errros.
cummulation (float) – Sum of squared errors distances.
- num_errors: int
- cummulation: float
- to_scalar()
Scalar representation of this artifact’s content.
Used to evaluate premature stopping criteria for the underlying evaluation.
- Returns
Scalar floating-point representation. None if a conversion to scalar is impossible.
- Return type
Optional[float]
- class RootMeanSquareEvaluation(pcl, ground_truth)
Bases:
Evaluation
Result of a single root mean squre evaluation.
- artifact()
Generate an artifact from this evaluation.
Returns: The evaluation artifact.
- Return type
- class RootMeanSquareErrorResult(grid, scalar_results, evaluator, plot_surface=True)
Bases:
ProcessedScalarEvaluationResult
Result of a root mean square error evaluation.
- plot_surface: bool
- class RootMeanSquareError(receiving_radar, radar_channel=None)
Bases:
RadarEvaluator
Root mean square estimation error of point detections.
- Parameters
receiving_radar (Radar) – Modem detecting radar in case of a target.
radar_channel (RadarChannel) – Radar channel containing a desired target.
- evaluate()
Evaluate the state of an investigated object.
Implements the process of extracting an arbitrary performance indicator, represented by the returned
Artifact
\(X_m\). :returns: Artifact \(X_m\) resulting from the evaluation. :rtype: Artifact
- property title: str
Long string representation of this evaluator.
Used as plot title.
- Returns
String representation
- Return type
str
- property abbreviation: str
Short string representation of this evaluator.
Used as a label for console output and plot axes annotations.
- Returns
String representation
- Return type
str
- generate_result(grid, artifacts)
Generates an evaluation result from the artifacts collected over the whole simulation grid.
- Parameters
grid (List[GridDimension]) – The Simulation grid.
artifacts (np.ndarray) – Numpy object array whose dimensions represent grid dimensions.
- Return type
- Returns
The evaluation result.