DES algorithm in VNC authentication with Python
DES algorithm in VNC authentication with Python
I am trying to connect with a VNC server (Vino) using VNC authentication. Reading the RFC 6143 - The Remote Framebuffer Protocol, we can see how VNC authentication works.
It is using a challenge-response protocol where the server sends a challenge of 16 bytes and the client answer with the challenge encrypted with his key. The encryption is using DES algorithm.
Using TigerVNC and Wireshark I have sniffed this process in order to do this process with Python.
E.g.
This is the traffic which I have sniffed with Wireshark between Vino and TigerVNC. It works.
Server Client
|------------------ Server protocol version: 003.007 ----------------->|
|<----------------- Client protocol version: 003.007 --------------|
|------------------ Security types: 2 --------------------->|
|<----------------- Security type selected: VNC (2) ------------------|
|------------------- Authentication result: OK --------->|
|----- Authentication challenge: b4a7257a443426527dd9d987fa6b099f ----->|
|<---- Authentication response: 4838c102d8cbb1decd38ecdbec533bc7 ------|
The challenge and response are bytes, not hex strings.
But when I encrypt the authentication challenge with Python, I got different results. I have tryed using differents block cipher mode of operations, without success.
E.g.
>>> from pydes import des
>>> challenge = "xb4xa7x25x7ax44x34x26x52x7dxd9xd9x87xfax6bx09x9f"
>>> key = "testingg"
>>> d = des()
>>> ciphered = d.encrypt(key,challenge,padding=True)
>>> import binascii
>>> ciphered = d.encrypt(key,challenge)
>>> binascii.hexlify(ciphered)
'4f16bc072bf34903e753b3f968b1aa56'
Or with another Python module:
>>> import pyDes
>>> des = pyDes.des("testingg")
>>> challenge = "xb4xa7x25x7ax44x34x26x52x7dxd9xd9x87xfax6bx09x9f"
>>> e = des.encrypt(challenge)
>>> binascii.hexlify(e)
'4f16bc072bf34903e753b3f968b1aa56'
>>> binascii.hexlify(des.decrypt(e))
'b4a7257a443426527dd9d987fa6b099f'
Am I missing something? The RFC doesn't show anything about encryption modes or IV vectors..
Moreover, I have shows source code of TigerVNC, which is wrote in Java, and It is not appers to have any special thing about the challenge encryption.
VNC would be compatible with makekey using crypt() and the low-order 7 bits of an 8 character key to provide 56 bits of key to prevent entropy loss. You could use crypt() for DES ECB by using a shifted input to setkey() and initializing the E permutation with a crypt call with a null salt then using encrypt() with edflag set for encrypt or decrypt. After demonstrating this in December 1990 the UCB crypt() had the edflag ignored preventing decryption for ITAR. Unnecessary in retrospect see Bernstein v. United States, code is speech. Now crypt() calls a DES implementation, still shifting.
– user1155120
Sep 14 '18 at 22:10
0
Thanks for contributing an answer to Stack Overflow!
But avoid …
To learn more, see our tips on writing great answers.
Required, but never shown
Required, but never shown
By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.
The processing of the DES key is non-standard in VNC. Instead of ignoring the low-order bit in each byte, VNC ignores the high-order bit.
– James K Polk
Sep 14 '18 at 11:49