Private Address Inconsistent Results

[jhughesoti]

I can't pinpoint to a specific release or commit but I have noticed some inconsistency with the private address test in the ipaddr module.

What I have found is that I can run DAQ and get an output that will return a failure for private address test and then I can run it again a few more times and maybe on the 2nd, 3rd, 4th, etc. I get a pass, which is weird. If I look even closer at these individual results, I will see that maybe in the first run it failed on the second IP range but then on the 2nd it passes that range but fails on the 3rd range. This tells me the device is completely capable of accepting these ranges but there is some issue with how the test is either providing them or an issue with timing on how it is validating. Additionally, I have found in the logs what appears to be a passing result in the individual tests but still fails in the final result.

In this output, you see the IP change to 10.99.246.73 and then fail a ping but then after the timeout, a ping again and it passes. The device fails the private address but is receiving all the addresses.

Apr 28 21:22:13 ipaddr.ipaddr01 INFO Device f833312cd6a4 running dhcp change test
Apr 28 21:22:13 ipaddr.ipaddr01 INFO Device f833312cd6a4 setting dhcp timeout at 2022-04-28 21:30:33.717266
Apr 28 21:22:21 ipaddr.ipaddr01 INFO Device f833312cd6a4 ip notification 10.99.246.73 (done)
Apr 28 21:22:32 ipaddr.ipaddr01 INFO Ping to 10.99.246.73, result False
Apr 28 21:22:32 ipaddr.ipaddr01 INFO Device f833312cd6a4 running analyze results
Apr 28 21:22:32 ipaddr.ipaddr01 INFO Device f833312cd6a4 setting dhcp timeout at 2022-04-28 21:30:52.531365
Apr 28 21:22:34 ipaddr.ipaddr01 INFO Device f833312cd6a4 ip notification 10.99.246.73 (done)
Apr 28 21:22:35 ipaddr.ipaddr01 INFO Ping to 10.99.246.73, result True

On a subsequent test, with no change to any configurations on the DAQ system or the device, this passes.

May 11 23:17:03 ipaddr.ipaddr01 INFO Device f833312cd6a4 running ip change test
May 11 23:17:03 ipaddr.ipaddr01 INFO Device f833312cd6a4 setting dhcp timeout at 2022-05-11 23:25:23.189214
May 11 23:21:18 ipaddr.ipaddr01 INFO Device f833312cd6a4 ip notification 10.99.246.72 (long)
May 11 23:21:19 ipaddr.ipaddr01 INFO Ping to 10.99.246.72, result True
May 11 23:21:19 ipaddr.ipaddr01 INFO Device f833312cd6a4 running dhcp change test
May 11 23:21:19 ipaddr.ipaddr01 INFO Device f833312cd6a4 setting dhcp timeout at 2022-05-11 23:29:39.448075
May 11 23:21:32 ipaddr.ipaddr01 INFO Device f833312cd6a4 ip notification 10.99.246.73 (done)
May 11 23:21:34 ipaddr.ipaddr01 INFO Ping to 10.99.246.73, result True
May 11 23:21:34 ipaddr.ipaddr01 INFO Device f833312cd6a4 running analyze results
May 11 23:21:34 ipaddr.ipaddr01 INFO Device f833312cd6a4 setting dhcp timeout at 2022-05-11 23:29:54.526310
May 11 23:21:38 ipaddr.ipaddr01 INFO Device f833312cd6a4 ip notification 10.99.246.73 (done)
May 11 23:21:49 ipaddr.ipaddr01 INFO Ping to 10.99.246.73, result True

I am currently having to run tests against the ipaddr module multiple times and inspect the logs to get the result due to this inconsistency since the test result isn't necessarily reliable.

[henry54809]

What are the different ip ranges to be tested? It looked like all the IP notifications were from 10.99.246.73, or 10.99.246.72

…

On Thu, May 12, 2022 at 9:29 AM jhughesbiot ***@***.***> wrote: I can't pinpoint to a specific release or commit but I have noticed some inconsistency with the private address test in the ipaddr module. What I have found is that I can run DAQ and get an output that will return a failure for private address test and then I can run it again a few more times and maybe on the 2nd, 3rd, 4th, etc. I get a pass, which is weird. If I look even closer at these individual results, I will see that maybe in the first run it failed on the second IP range but then on the 2nd it passes that range but fails on the 3rd range. This tells me the device is completely capable of accepting these ranges but there is some issue with how the test is either providing them or an issue with timing on how it is validating. Additionally, I have found in the logs what appears to be a passing result in the individual tests but still fails in the final result. In this output, you see the IP change to 10.99.246.73 and then fail a ping but then after the timeout, a ping again and it passes. The device fails the private address but is receiving all the addresses. Apr 28 21:22:13 ipaddr.ipaddr01 INFO Device f833312cd6a4 running dhcp change test Apr 28 21:22:13 ipaddr.ipaddr01 INFO Device f833312cd6a4 setting dhcp timeout at 2022-04-28 21:30:33.717266 Apr 28 21:22:21 ipaddr.ipaddr01 INFO Device f833312cd6a4 ip notification 10.99.246.73 (done) Apr 28 21:22:32 ipaddr.ipaddr01 INFO Ping to 10.99.246.73, result False Apr 28 21:22:32 ipaddr.ipaddr01 INFO Device f833312cd6a4 running analyze results Apr 28 21:22:32 ipaddr.ipaddr01 INFO Device f833312cd6a4 setting dhcp timeout at 2022-04-28 21:30:52.531365 Apr 28 21:22:34 ipaddr.ipaddr01 INFO Device f833312cd6a4 ip notification 10.99.246.73 (done) Apr 28 21:22:35 ipaddr.ipaddr01 INFO Ping to 10.99.246.73, result True On a subsequent test, with no change to any configurations on the DAQ system or the device, this passes. May 11 23:17:03 ipaddr.ipaddr01 INFO Device f833312cd6a4 running ip change test May 11 23:17:03 ipaddr.ipaddr01 INFO Device f833312cd6a4 setting dhcp timeout at 2022-05-11 23:25:23.189214 May 11 23:21:18 ipaddr.ipaddr01 INFO Device f833312cd6a4 ip notification 10.99.246.72 (long) May 11 23:21:19 ipaddr.ipaddr01 INFO Ping to 10.99.246.72, result True May 11 23:21:19 ipaddr.ipaddr01 INFO Device f833312cd6a4 running dhcp change test May 11 23:21:19 ipaddr.ipaddr01 INFO Device f833312cd6a4 setting dhcp timeout at 2022-05-11 23:29:39.448075 May 11 23:21:32 ipaddr.ipaddr01 INFO Device f833312cd6a4 ip notification 10.99.246.73 (done) May 11 23:21:34 ipaddr.ipaddr01 INFO Ping to 10.99.246.73, result True May 11 23:21:34 ipaddr.ipaddr01 INFO Device f833312cd6a4 running analyze results May 11 23:21:34 ipaddr.ipaddr01 INFO Device f833312cd6a4 setting dhcp timeout at 2022-05-11 23:29:54.526310 May 11 23:21:38 ipaddr.ipaddr01 INFO Device f833312cd6a4 ip notification 10.99.246.73 (done) May 11 23:21:49 ipaddr.ipaddr01 INFO Ping to 10.99.246.73, result True I am currently having to run tests against the ipaddr module multiple times and inspect the logs to get the result due to this inconsistency since the test result isn't necessarily reliable. — Reply to this email directly, view it on GitHub <#956>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABQ5NYKNUZKXYZTCNRL7DMDVJUWW3ANCNFSM5VY2FUVQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[jhughesoti]

The ranges used are all the default ranges defined. https://github.com/faucetsdn/daq/blob/master/resources/setups/common/base_config.json#L7

This was just an example from the most recent device being qualified to show the inconsistencies over the same range. Other devices and other runs of DAQ on this device show failures and success over different ranges so there's not just one range where it happens, it's the whole test that doesn't act consistently.

[evensonbryan]

I think I found the cause of this issue:

daq/subset/ipaddr/dhcp_tests.py

Line 66 in 8d279c4

if not packet[DHCP].options[0][1] == DHCP_REQUEST:

This line assumes that the 'message-type' option is always the first option in the DHCP option list. From my testing, this is not the case. I changed line 66 to:

if not _get_dhcp_option(packet, 'message-type') == DHCP_REQUEST:

and the private address test is now passing for me. If this looks good, I can make a pull request for the change.

On a related note, the same error is present in the function _get_dhcp_type() in the same file. However, this function is never used. From what I can tell in the Git history, _get_dhcp_type() has never been used. Should _get_dhcp_type() be removed? If so, would be okay to include it in the same pull request as the other change?