Support for partial decoding when input message is truncated or when decoding fails.

[dbressan2]

Hi
When an ASN.1 UPER decoding fails, end user is given very little information. Typically, stderr just contains "bitlen overflow: 4, max 0". It is common to get only a partial logging of a message, and decoding it would be useful.

Maybe it is already supported, but I do not know how to activate it. Ideally, I would like to retrieve the decoding results up to the point where decoding failed, either due to message not being properly formatted or being truncated.

Wireshark RRC decoder for 3GPP supports such a feature. Wireshark dissector prints all partial decoding results followed by "[Expert Info (Error/Malformed): Malformed Packet (Exception occurred)]", highlighting the information element where decoding failed.

This feature would be useful for debugging (from which field does decoding fails ? ) and for understanding what a log contains when e.g. only the first 100 bytes of a message are logged due to memory constraints.

Thanks in advance

[mitshell]

Hello,
Currently, an exception is raised when the decoding fails. While most of the decoding errors are raising a dedicated exception from pycrate, there is no guarantee that such error would always be caught and raised properly. This is handled this way as all decoding methods in pycrate are returning None, and make the decoded value available under the ._val attribute. This can be obtained with the get_val() method then. There is no return code from the decoding methods, that could enable to distinguish a decoding that went perfectly well, partially well, or totally wrong...

I admit the raised exceptions are not always self-explanatory, and the provided information is scarse. If the decoding error is caught properly as part of the object's runtime, the object's name should be provided at least. It's not always the case, as some processing is done in object's independent classes: there, you don't get the object's name when it fails. What I do generally, is just use the Python debugger when I get an error, and trace back the object from which it originated. Clearly a more uniform and self-explanatory exception handling would certainly be nice for the ASN.1 runtime: this remains a lot of work, and contributions are welcomed.

Finally, if we want to complete the decoding without raising despite of errors, we would need to change the API through which we call objects' decoding methods. That could enabled to return half-decoded values, but that would also be a breaking change for applications using pycrate's ASN.1 runtime. Thus, I'm quite puzzled about this.

For your specific concern, there may be a quick solution by catching any exceptions raised during decoding, and backtracking from the object in which the decoding raised, up to the top-level one, and collecting values available in every ._val attributes.
I hope this helps.

[dbressan2]

Thank you very much for your detailed answer.
Using the get_val method is definitely informative when decoding fails due to constraint error.
However, it does not really help for shortened messages. To put things in perspective, here is a simple example with one octet missing for an NR-5G UL-DCH Message:

input:
58 a0 08 00 47

pycrate output:
bitlen overflow: 10, max 3

wireshark output:
NR Radio Resource Control (RRC) protocol
UL-DCCH-Message
message: c1 (0)
c1: ueAssistanceInformation (11)
ueAssistanceInformation
criticalExtensions: ueAssistanceInformation (0)
ueAssistanceInformation
nonCriticalExtension
nonCriticalExtension
nonCriticalExtension
propagationDelayDifference-r17: 1 item
Item 0
[Malformed Packet: NR RRC]
[Expert Info (Error/Malformed): Malformed Packet (Exception occurred)]
[Malformed Packet (Exception occurred)]
[Severity level: Error]
[Group: Malformed]

asn1c output:
Decode failed past byte 0: Unexpected end of input
=== Partial Decoding Results (RC_WMORE) ===
UL-DCCH-Message ::= {
message: c1: ueAssistanceInformation: UEAssistanceInformation ::= {
criticalExtensions: ueAssistanceInformation: UEAssistanceInformation-IEs ::= {
nonCriticalExtension: UEAssistanceInformation-v1540-IEs ::= {
nonCriticalExtension: UEAssistanceInformation-v1610-IEs ::= {
nonCriticalExtension: UEAssistanceInformation-v1700-IEs ::= {
propagationDelayDifference-r17: PropagationDelayDifference-r17 ::= {
}
}
}
}
}
}
}
=== End of Partial Results ===

I fully understand it is certainly a feature request or enhancement request, rather than a product defect. But it would be a very useful feature as some devices only perform partial RRC logging due to memory constraints and it would be very useful to get the partial decoding information.

Many thanks as always for developing and maintaining the pycrate library.