Radar Evaluation#
This module introduces several evaluators for performance indicators in radar detection.
Refer to the PyMonte documentation for a detailed introduction to the concept of
Evaluators
.
The implemented RadarEvaluator
all inherit from the identically named common
base which gets initialized by selecting one Modem
, whose performance will be
evaluated and one RadarChannel
instance, containing the ground
truth.
The currently considered performance indicators are
Evaluator |
Artifact |
Performance Indicator |
---|---|---|
Probability of detecting the target at the right bin |
||
Probability of detection versus probability of false alarm |
||
:class`.RootMeanSquareError` |
Root mean square error of point detections |
Configuring RadarEvaluators
to evaluate the radar detection of
Modem
instances is rather straightforward:
# Create two separate modem instances
modem = Modem()
channel = RadarChannel()
# Create a radar evaluation as an evaluation example
radar_evaluator = DetectionProbEvaluator(modem, channel)
# Extract evaluation
radar_evaluation = radar_evaluator.evaluate()
# Visualize evaluation
radar_evaluation.plot()
- class RadarEvaluator(receiving_radar, radar_channel=None)#
Bases:
Evaluator
,ABC
Base class for evaluating sensing performance.
- Parameters:
receiving_radar (Radar) – nRadar under test.
radar_channel (RadarChannelBase) – Radar channel modeling a desired target.
- Raises:
ValueError – If the receiving radar is not an operator of the radar_channel receiver.
- property receiving_radar: Radar#
Radar detector with target present.
- Returns:
Handle to the receiving radar, when target is present.
- Return type:
Modem
- property radar_channel: RadarChannelBase#
The considered radar channel.
- generate_result(grid, artifacts)#
Generates an evaluation result from the artifacts collected over the whole simulation grid.
- Parameters:
grid (Sequence[GridDimension]) – The Simulation grid.
artifacts (np.ndarray) – Numpy object array whose dimensions represent grid dimensions.
- Return type:
- Returns:
The evaluation result.
- class DetectionProbArtifact(artifact)#
Bases:
ArtifactTemplate
[bool
]Artifact of a detection probability evaluation for a radar detector.
- Parameters:
artifact (AT) – Artifact value.
- class DetectionProbabilityEvaluation(evaluation)#
Bases:
EvaluationTemplate
[bool
]- artifact()#
Generate an artifact from this evaluation.
Returns: The evaluation artifact.
- Return type:
- class DetectionProbEvaluator(receiving_radar)#
Bases:
RadarEvaluator
,Serializable
Evaluate detection probability at a radar detector, considering any bin, i.e., detection is considered if any bin in the radar cube is above the threshold
- Parameters:
receiving_radar (Radar) – Radar detector
- yaml_tag: Optional[str] = 'DetectionProbEvaluator'#
YAML serialization tag
- property abbreviation: str#
Short string representation of this evaluator.
Used as a label for console output and plot axes annotations.
- Returns:
String representation
- Return type:
str
- property title: str#
Long string representation of this evaluator.
Used as plot title.
- Returns:
String representation
- Return type:
str
- generate_result(grid, artifacts)#
Generates an evaluation result from the artifacts collected over the whole simulation grid.
- Parameters:
grid (Sequence[GridDimension]) – The Simulation grid.
artifacts (np.ndarray) – Numpy object array whose dimensions represent grid dimensions.
- Return type:
- Returns:
The evaluation result.
- class RocArtifact(h0_value, h1_value)#
Bases:
Artifact
Artifact of receiver operating characteristics (ROC) evaluation
- Parameters:
h0_value (float) – Measured value for null-hypothesis (H0), i.e., noise only
h1_value (float) – Measured value for alternative hypothesis (H1)
- property h0_value: float#
- property h1_value: float#
- class RocEvaluation(cube_h0, cube_h1)#
Bases:
Evaluation
Evaluation of receiver operating characteristics (ROC)
-
data_h0:
ndarray
#
-
data_h1:
ndarray
#
- artifact()#
Generate an artifact from this evaluation.
Returns: The evaluation artifact.
- Return type:
-
data_h0:
- class RocEvaluationResult(grid, detection_probabilities, false_alarm_probabilities, title='Receiver Operating Characteristics')#
Bases:
EvaluationResult
Final result of an receive operating characteristcs evaluation.
- Parameters:
grid (Sequence[GridDimension]) – Grid dimensions of the evaluation result.
detection_probabilities (np.ndarray) – Detection probabilities for each grid point.
false_alarm_probabilities (np.ndarray) – False alarm probabilities for each grid point.
title (str, optional) – Title of the evaluation result.
- property title: str#
Title of the visualizable.
Returns: Title string.
- to_array()#
Convert the evaluation result raw data to an array representation.
Used to store the results in arbitrary binary file formats after simulation execution.
- Return type:
ndarray
- Returns:
The array result representation.
- class ReceiverOperatingCharacteristic(radar, radar_channel=None, num_thresholds=101)#
Bases:
RadarEvaluator
,Serializable
Evaluate the receiver operating characteristics for a radar operator.
- Parameters:
radar (Radar) – Radar under test.
radar_channel (RadarChannelBase, optional) – Radar channel containing a desired target. If the radar channel is not specified, the
evaluate()
routine will not be available.num_thresholds (int, optional) – Number of different thresholds to be considered in ROC curve
- yaml_tag: Optional[str] = 'ROC'#
YAML serialization tag.
- evaluate()#
Evaluate the state of an investigated object.
Implements the process of extracting an arbitrary performance indicator, represented by the returned
Artifact
\(X_m\). :returns: Artifact \(X_m\) resulting from the evaluation. :rtype: Artifact
- property abbreviation: str#
Short string representation of this evaluator.
Used as a label for console output and plot axes annotations.
- Returns:
String representation
- Return type:
str
- property title: str#
Long string representation of this evaluator.
Used as plot title.
- Returns:
String representation
- Return type:
str
- generate_result(grid, artifacts)#
Generates an evaluation result from the artifacts collected over the whole simulation grid.
- Parameters:
grid (Sequence[GridDimension]) – The Simulation grid.
artifacts (np.ndarray) – Numpy object array whose dimensions represent grid dimensions.
- Return type:
- Returns:
The evaluation result.
- classmethod From_Scenarios(h0_scenario, h1_scenario, h0_operator=None, h1_operator=None)#
- Return type:
- classmethod From_HDF(file, h0_campaign='h0_measurements', h1_campaign='h1_measurements')#
Compute an ROC evaluation result from a savefile.
- Parameters:
file (Union[str, File]) – Savefile containing the measurements. Either as file system location or h5py File handle.
h0_campaign (str, optional) – Campaign identifier of the h0 hypothesis measurements. By default, h0_measurements is assumed.
h1_campaign (str, optional) – Campaign identifier of the h1 hypothesis measurements. By default, h1_measurements is assumed.
- Return type:
Returns: The ROC evaluation result.
- class RootMeanSquareArtifact(num_errors, cummulation)#
Bases:
Artifact
Artifact of a root mean square evaluation
- Parameters:
num_errors (int) – Number of errros.
cummulation (float) – Sum of squared errors distances.
- to_scalar()#
Scalar representation of this artifact’s content.
Used to evaluate premature stopping criteria for the underlying evaluation.
- Returns:
Scalar floating-point representation. None if a conversion to scalar is impossible.
- Return type:
Optional[float]
- property num_errors: int#
Number of cummulated errors
- property cummulation: float#
Cummulated squared error
- class RootMeanSquareEvaluation(pcl, ground_truth)#
Bases:
Evaluation
Result of a single root mean squre evaluation.
- artifact()#
Generate an artifact from this evaluation.
Returns: The evaluation artifact.
- Return type:
- class RootMeanSquareErrorResult(grid, scalar_results, evaluator, plot_surface=True)#
Bases:
ScalarEvaluationResult
Result of a root mean square error evaluation.
- Parameters:
grid (Sequence[GridDimension]) – Simulation grid.
scalar_results (np.ndarray) – Scalar results generated from collecting samples over the simulation grid.
evaluator (Evaluator) – The evaluator generating the results.
plot_surface (bool, optional) – Enable surface plotting for two-dimensional grids. Enabled by default.
- class RootMeanSquareError(receiving_radar, radar_channel=None)#
Bases:
RadarEvaluator
Root mean square estimation error of point detections.
- Parameters:
receiving_radar (Radar) – nRadar under test.
radar_channel (RadarChannelBase) – Radar channel modeling a desired target.
- Raises:
ValueError – If the receiving radar is not an operator of the radar_channel receiver.
- evaluate()#
Evaluate the state of an investigated object.
Implements the process of extracting an arbitrary performance indicator, represented by the returned
Artifact
\(X_m\). :returns: Artifact \(X_m\) resulting from the evaluation. :rtype: Artifact
- property title: str#
Long string representation of this evaluator.
Used as plot title.
- Returns:
String representation
- Return type:
str
- property abbreviation: str#
Short string representation of this evaluator.
Used as a label for console output and plot axes annotations.
- Returns:
String representation
- Return type:
str
- generate_result(grid, artifacts)#
Generates an evaluation result from the artifacts collected over the whole simulation grid.
- Parameters:
grid (Sequence[GridDimension]) – The Simulation grid.
artifacts (np.ndarray) – Numpy object array whose dimensions represent grid dimensions.
- Return type:
- Returns:
The evaluation result.