Topic: DTLS negotiation fails if packets arrive in wrong order

We are evaluating wolfSSL embedded SSL version 2.4.0 for DTLS and so far it works great but yesterday we found something strange. It looks like if the UDP packets does not arrive in the expected order to the client, the client fails to complete the negotiation.

The UDP packets that are sent between the client (c) and the server (s) are in our test case:
1. c->s client_hello
2. s->c hello_verify_request
3. c->s client_hello
4. s->c server_hello
    s->c certificate
    s->c certificate_request
    s->c server_hello_done
5. c->s certificate
... (the rest of the negiotion is not included here)

And the above works very well. The problem is if some packets in the same flight in step 4 arrives to the client in wrong order. As an example, if the packets in step 4 arrives in this order at the client:
4. server_hello_done
    certificate_request
    certificate
    server_hello 

Then, the client will fail to continue the negotiation process and will never send its certificate back to the server. Can you reproduce this scenario at your place? Is this a known issue? Is it something that will be fixed?

Share

Re: DTLS negotiation fails if packets arrive in wrong order

Are you using wolfSSL embedded SSL as both test server and client? Does the server retry sending its flight?

Since there is a retry mechanism in place for DTLS handshaking, we used it to take care of out of order handshake messages; we drop messages with out of order handshake sequence numbers and wait for the retry mechanism to resend the message when its response wasn't received by the other end. In the case you show, the client should have retried its second Client Hello as it didn't get the expected response from the server.

We are planning on some refactoring in the DTLS code in the near future and will take a look into this.

Re: DTLS negotiation fails if packets arrive in wrong order

According to my experiment, if "sever hello done" is before "server hello", the dtls client would stop replying any message when it goes to connectstate HELLO_AGAIN_REPLY, the neededState is set to SERVER_HELLODONE_COMPLETE,  in the mean while server is retry to send those message again and again...

Share

Re: DTLS negotiation fails if packets arrive in wrong order

We are looking into some improvements to DTLS starting this week.

Could you possibly send me the code you are testing with? It will make a good test case. Our TLS will reject a session with an out of order error if the handshake messages are received out of order. In DTLS, we are checking the handshake sequence numbers and rejecting out of order messages, and performing retries as needed. If the sequence number is correct, but the actual message is in the wrong order, we aren't rejecting it as out of order. This looks like something that needs to be fixed. Thank you for catching this.