NTLMSSP (NT LAN Manager (NTLM) Security Support Provider) is a binary messaging protocol used by the Microsoft Security Support Provider Interface (SSPI) to facilitate NTLM challenge-response authentication and to negotiate integrity and confidentiality options. NTLMSSP is used wherever SSPI authentication is used including Server Message Block / CIFS extended security authentication, HTTP Negotiate authentication (e.g. IIS with IWA turned on) and MSRPC services.
The NTLMSSP and NTLM challenge-response protocol have been documented in Microsoft's Open Protocol Specification.
References
MS-NLMP - NT LAN MANAGER (NTLM) Authentication Protocol Specification Archived 2008-04-11 at the Wayback Machine https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-nlmp/b38c36ed-2804-4868-a9ff-8dd3182128e4 ↩