Way back when, I was involved in trying to obtain passwords for a Novell NetWare 3.12 server. I won’t go into details here, suffice to say that the topic has always interested me – sufficiently to return to it 30-ish years later and write down an algorithm description.
The research presented here is based on my independent disassembly of the embedded server.nlm in NetWare 3.12. I have compared these findings to ncpfs 2.2.0, mars_nwe-0.99.p121 and itsme’s sources, and believe the information as presented here to be correct. However, if you spot mistakes or omissions please let me know!
Note that this only covers password storage and client login. Password changes are not covered in this entry as this is a separate mechanism.
You can find code implementing these algorithms, including test vectors, at https://github.com/zhmu/nw-tools/blob/main/nw-crypt.c.
Introduction
NetWare 3.x stores passwords in the bindery, which is an internal database containing objects (users, groups) with properties which can have a value. Every user should have a password property, which can only be accessed by the server itself. This password property contains a password hash as well as the password length.
Passwords are not stored in plain-text, but in a hashed form. The hash function is used both for password hashing as well as login credentials.
NetWare hash function
The NetWare-specific hash function is one-way and cannot be inverted. It uses the following variables:
- Salt (
salt): 4 bytes, commonly referred to as a key - Input (
in): 32 bytes - Output (
out): 16 bytes - Temporary storage (
temp): 32 bytes - Carry value (
last): 1 byte, initialized to 0 - Shuffle keys (
keys_table): 32 bytes, provided as an appendix - Hash nibble table (
nibble_table): 256 bytes, provided as an appendix
The operation of nw_hash(salt, in) is the following:
- Apply salt to input data, by Exclusing Or-ing the input with the corresponding salt value, for n in [ 0 .. 31 ]:
temp[n] = in[n]ExclusiveOrsalt[n % 4] - Two rounds
Execute step 3 twice - Shuffle the content on a per-byte basis, for index in [ 0 .. 31 ]:
v = temp[(last + index) % 16] - key_table[index]
new_value = (temp[index] + last) ExclusiveOr v
last = last + new_value
temp[index] = new_value - Combine the 32-byte temporary data to a 16-byte result, for index in [ 0 .. 15 ]:
out[index] = nibble_table[temp[index * 2 + 0] BitwiseOr
(nibble_table[temp[index * 2 + 1] ShiftLeft 4) - Return
out
The addition and subtraction used in step 3 must be performed with wrapping, i.e. 255 + 1 = 0 and 0 - 1 = 255.
Input stretching
The hashing algorithm just described always takes a 32-byte input. This means the password must be transformed in such a way so that it occupies exactly 32 bytes. I’ve dubbed this step input stretching as it is similar in spirit to key stretching. We have the following values:
- Input (
input): arbitrary bytes with a known length - Input length (
in_len): Length ofinput, in bytes - Output (
out): 32 bytes - Input position (
input_pos): Offset of input byte to process
The operation of stretch_input(in, in_len) is as follows:
- Discard trailing zero-bytes from the input
while (in_len > 0 && in[in_len - 1] == 0)
in_len = in_len - 1; - Set the initial output to zero, for n in [ 0.. 31 ]:
output[n] = 0 - While the input exceeds 32 bytes, Exclusive Or the input bytes to the output.
For n in [ 0 .. 31 ]:out[n] = out[n] ExclusiveOr input[0]discard input[0] from the input stream
Decrease the input length by 32 - Process input data, for n in [ 0.. 31 ], input_pos = 0
if position input_pos is present in the input:
out[n] = out[n]ExclusiveOrin[input_pos]
input_pos = input_pos + 1elseout[n] = out[n] ExclusiveOr key_table[input_pos]
input_pos = 0 - Return
out
Calculating a password hash for a given bindery object
We have the following values:
- Object ID (
object_id), a 32-bit unsigned value uniquely identifying the bindery object - Password (
password): The ASCII password to use, in UPPERCASE. - Password length (
password_length): Length ofpassword, in bytes - Output (
output): 16 bytes - Salt (
salt): 4-byte input salt, generated fromobject_id
The operation of hash_object_password(object_id, pwd) is as follows:
- Generate the salt by encoding the object id in a 4-byte array, big-endian
- Invoke
stretch_input(password, password_length, expanded_in) - Invoke
nw_hash(salt, expanded_in, out) - Return
out
Server password storage
The result of hash_object_password() is used for this step. The resulting 16 bytes will be stored in the password property of the object. Byte 17 will be the length of the password. All other bytes are set to zero.
Client password encryption
Password hashes are not sent in plaintext over the network. There is a 8-byte login key involved, which is sent from the server to the client and used by both of them.
We have the following values:
- Login key (
key): a 8 byte pseudo-random value, generated by the server - Input hash (
input), 16 bytes input value - Output (
output): 8 bytes - Temporary expanded input storage (
expanded_input): 32 bytes - Temporary data (
temp): 32 bytes
The operation of nw_encrypt(key, input, output) is as follows:
- Expand input to 32 bytes
Invokestretch_input(input, 16), store result inexpanded_input. - Calculate NetWare hash for each subkey:
temp[0..15] = nw_hash(key[0..3], expanded_input)
temp[16..31] = nw_hash(key[4..7], expanded_input) - Combine the 32-byte resulting values to a 8-byte value, for n in [ 0 .. 7 ]:
out[n] = temp[n] ExclusiveOr temp[31 - n] ExclusiveOr temp[15 - n] ExclusiveOr temp[16 + n] - Return
out
Putting it all together
The image below illustrates how a plaintext password is set, and how the client login mechanism works. Colours are used to highlight values that are always identical, given a correct password. If the password mismatches, the red and blue values will differ.
Appendix: tables
nibble_table is used to combine two bytes. The byte values are looked up, and the first byte maps to the high nibble and the second byte maps to the low nibble (refer to step 4 of nw_hash):
const uint8_t nibble_table[256] = {
0x7, 0x8, 0x0, 0x8, 0x6, 0x4, 0xE, 0x4, 0x5, 0xC, 0x1, 0x7, 0xB, 0xF, 0xA, 0x8,
0xF, 0x8, 0xC, 0xC, 0x9, 0x4, 0x1, 0xE, 0x4, 0x6, 0x2, 0x4, 0x0, 0xA, 0xB, 0x9,
0x2, 0xF, 0xB, 0x1, 0xD, 0x2, 0x1, 0x9, 0x5, 0xE, 0x7, 0x0, 0x0, 0x2, 0x6, 0x6,
0x0, 0x7, 0x3, 0x8, 0x2, 0x9, 0x3, 0xF, 0x7, 0xF, 0xC, 0xF, 0x6, 0x4, 0xA, 0x0,
0x2, 0x3, 0xA, 0xB, 0xD, 0x8, 0x3, 0xA, 0x1, 0x7, 0xC, 0xF, 0x1, 0x8, 0x9, 0xD,
0x9, 0x1, 0x9, 0x4, 0xE, 0x4, 0xC, 0x5, 0x5, 0xC, 0x8, 0xB, 0x2, 0x3, 0x9, 0xE,
0x7, 0x7, 0x6, 0x9, 0xE, 0xF, 0xC, 0x8, 0xD, 0x1, 0xA, 0x6, 0xE, 0xD, 0x0, 0x7,
0x7, 0xA, 0x0, 0x1, 0xF, 0x5, 0x4, 0xB, 0x7, 0xB, 0xE, 0xC, 0x9, 0x5, 0xD, 0x1,
0xB, 0xD, 0x1, 0x3, 0x5, 0xD, 0xE, 0x6, 0x3, 0x0, 0xB, 0xB, 0xF, 0x3, 0x6, 0x4,
0x9, 0xD, 0xA, 0x3, 0x1, 0x4, 0x9, 0x4, 0x8, 0x3, 0xB, 0xE, 0x5, 0x0, 0x5, 0x2,
0xC, 0xB, 0xD, 0x5, 0xD, 0x5, 0xD, 0x2, 0xD, 0x9, 0xA, 0xC, 0xA, 0x0, 0xB, 0x3,
0x5, 0x3, 0x6, 0x9, 0x5, 0x1, 0xE, 0xE, 0x0, 0xE, 0x8, 0x2, 0xD, 0x2, 0x2, 0x0,
0x4, 0xF, 0x8, 0x5, 0x9, 0x6, 0x8, 0x6, 0xB, 0xA, 0xB, 0xF, 0x0, 0x7, 0x2, 0x8,
0xC, 0x7, 0x3, 0xA, 0x1, 0x4, 0x2, 0x5, 0xF, 0x7, 0xA, 0xC, 0xE, 0x5, 0x9, 0x3,
0xE, 0x7, 0x1, 0x2, 0xE, 0x1, 0xF, 0x4, 0xA, 0x6, 0xC, 0x6, 0xF, 0x4, 0x3, 0x0,
0xC, 0x0, 0x3, 0x6, 0xF, 0x8, 0x7, 0xB, 0x2, 0xD, 0xC, 0x6, 0xA, 0xA, 0x8, 0xD
};
key_table is used when transforming the input values (refer to step 3 of nw_hash), as well as additional input when stretching the input to 32 bytes (step 4 of stretch_input):
const uint8_t key_table[32] = {
0x48, 0x93, 0x46, 0x67, 0x98, 0x3D, 0xE6, 0x8D,
0xB7, 0x10, 0x7A, 0x26, 0x5A, 0xB9, 0xB1, 0x35,
0x6B, 0x0F, 0xD5, 0x70, 0xAE, 0xFB, 0xAD, 0x11,
0xF4, 0x47, 0xDC, 0xA7, 0xEC, 0xCF, 0x50, 0xC0
};
I haven’t found any specific patterns in nibble_table or key_table. They are hardcoded in both server.nlm and setpass.exe. However, perhaps there is a way to generate these tables by use of a function. If you manage to find it, let me know!