6.5.5 DTLS sessions

Because datagram TLS can operate over connections where the client cannot be reliably verified, functionality in the form of cookies, is available to prevent denial of service attacks to servers. GnuTLS requires a server to generate a secret key that is used to sign a cookie¹⁹. That cookie is sent to the client using gnutls_dtls_cookie_send, and the client must reply using the correct cookie. The server side should verify the initial message sent by client using gnutls_dtls_cookie_verify. If successful the session should be initialized and associated with the cookie using gnutls_dtls_prestate_set, before proceeding to the handshake.

int gnutls_key_generate (gnutls_datum_t * key, unsigned int key_size)

int gnutls_dtls_cookie_send (gnutls_datum_t * key, void * client_data, size_t client_data_size, gnutls_dtls_prestate_st * prestate, gnutls_transport_ptr_t ptr, gnutls_push_func push_func)

int gnutls_dtls_cookie_verify (gnutls_datum_t * key, void * client_data, size_t client_data_size, void * _msg, size_t msg_size, gnutls_dtls_prestate_st * prestate)

void gnutls_dtls_prestate_set (gnutls_session_t session, gnutls_dtls_prestate_st * prestate)

Note that the above apply to server side only and they are not mandatory to be used. Not using them, however, allows denial of service attacks. The client side cookie handling is part of gnutls_handshake.

Datagrams are typically restricted by a maximum transfer unit (MTU). For that both client and server side should set the correct maximum transfer unit for the layer underneath GnuTLS. This will allow proper fragmentation of DTLS messages and prevent messages from being silently discarded by the transport layer. The “correct” maximum transfer unit can be obtained through a path MTU discovery mechanism [RFC4821].

void gnutls_dtls_set_mtu (gnutls_session_t session, unsigned int mtu)

unsigned int gnutls_dtls_get_mtu (gnutls_session_t session)

unsigned int gnutls_dtls_get_data_mtu (gnutls_session_t session)