Once a session has been initialized and a network connection has been set up, TLS and DTLS protocols perform a handshake. The handshake is the actual key exchange.
session: is a
This function performs the handshake of the TLS/SSL protocol, and initializes the TLS session parameters.
The non-fatal errors expected by this function are:
GNUTLS_E_WARNING_ALERT_RECEIVED . When this function is called
for re-handshake under TLS 1.2 or earlier, the non-fatal error code
GNUTLS_E_GOT_APPLICATION_DATA may also be returned.
The former two interrupt the handshake procedure due to the transport
layer being interrupted, and the latter because of a "warning" alert that
was sent by the peer (it is always a good idea to check any
received alerts). On these non-fatal errors call this function again,
until it returns 0; cf.
gnutls_error_is_fatal() . In DTLS sessions the non-fatal error
GNUTLS_E_LARGE_PACKET is also possible, and indicates that
the MTU should be adjusted.
When this function is called by a server after a rehandshake request
under TLS 1.2 or earlier the
GNUTLS_E_GOT_APPLICATION_DATA error code indicates
that some data were pending prior to peer initiating the handshake.
Under TLS 1.3 this function when called after a successful handshake, is a no-op
and always succeeds in server side; in client side this function is
This function handles both full and abbreviated TLS handshakes (resumption).
For abbreviated handshakes, in client side, the
should be called prior to this function to set parameters from a previous session.
In server side, resumption is handled by either setting a DB back-end, or setting
up keys for session tickets.
GNUTLS_E_SUCCESS on a successful handshake, otherwise a negative error code.
session: is a
ms: is a timeout value in milliseconds
This function sets the timeout for the TLS handshake process
to the provided value. Use an
ms value of zero to disable
GNUTLS_DEFAULT_HANDSHAKE_TIMEOUT for a reasonable
default value. For the DTLS protocol, the more detailed
gnutls_dtls_set_timeouts() is provided.
This function requires to set a pull timeout callback. See
In GnuTLS 3.5.0 and later it is recommended to use gnutls_session_set_verify_cert for the handshake process to ensure the verification of the peer’s identity. That will verify the peer’s certificate, against the trusted CA store while accounting for stapled OCSP responses during the handshake; any error will be returned as a handshake error.
In older GnuTLS versions it is required to verify the peer’s certificate during the handshake by setting a callback with gnutls_certificate_set_verify_function, and then using gnutls_certificate_verify_peers3 from it. See Certificate authentication for more information.
void gnutls_session_set_verify_cert (gnutls_session_t session, const char * hostname, unsigned flags)
int gnutls_certificate_verify_peers3 (gnutls_session_t session, const char * hostname, unsigned int * status)