Bug 1296595 - libhivex: use on Windows 7 NTUSER.DAT causes damage that manifests itself with folder rename errors
libhivex: use on Windows 7 NTUSER.DAT causes damage that manifests itself wit...
Status: NEW
Product: Virtualization Tools
Classification: Community
Component: hivex (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Richard W.M. Jones
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2016-01-07 10:55 EST by Sebastian J. Bronner
Modified: 2016-06-15 08:41 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
working user hive before editing with libhivex (4.00 MB, application/octet-stream)
2016-01-07 10:55 EST, Sebastian J. Bronner
no flags Details
python program using libhivex, required for the testcase (5.95 KB, text/x-python)
2016-01-07 10:57 EST, Sebastian J. Bronner
no flags Details
user hive resulting from edit that causes folder rename errors (4.00 MB, application/octet-stream)
2016-01-07 10:58 EST, Sebastian J. Bronner
no flags Details

  None (edit)
Description Sebastian J. Bronner 2016-01-07 10:55:49 EST
Created attachment 1112486 [details]
working user hive before editing with libhivex

Description of problem:

After editing an NTUSER.DAT hive, taken from a Windows 7 user account, with libhivex, that user receives an error when renaming folders.

Version-Release number of selected component:

python3-hivex 1.3.11
libhivex0 1.3.11

How reproducible:

always for large hives (a few megabytes)

Steps to Reproduce:

1. Take the attached hive NTUSER.DAT.orig and python program registry-replace.

2. Run the following command:
   registry-replace -p '\\sbr' -r '\\sbr' NTUSER.DAT

3. The attached hive NTUSER.DAT.result is the result.

4. Install it in a Windows 7 user profile named sbr.

5. Log in as that user.

6. Rename a folder (or create a new folder specifying a name). The location doesn't seem to matter, but the My Documents folder is where I tested it.

Actual results:

The registry hive is successfully altered, the new values are visible in the user account. However, the user always faces the following message when renaming a folder (also applies when creating a new folder with a non-default name):

Rename Folder

An unexpected error is keeping you from renaming the folder. If you continue to receive this error, you can use the error code to search for help with this problem.

Error 0x80004005: Unknown Error

[Repeat] [Cancel]

After clicking on [Repeat] the folder is renamed (or created) successfully.

Expected results:

In addition to the hive being successfully altered, renaming (and creating) folders should continue to be possible without error messages popping up.

Additional info:

The library soandso manages to do this. Therefore, I am assuming it has something to do with some new way that hives are organized in Windows 7 and later. ...
Comment 1 Sebastian J. Bronner 2016-01-07 10:57 EST
Created attachment 1112487 [details]
python program using libhivex, required for the testcase
Comment 2 Sebastian J. Bronner 2016-01-07 10:58 EST
Created attachment 1112488 [details]
user hive resulting from edit that causes folder rename errors
Comment 3 Sebastian J. Bronner 2016-01-07 11:12:07 EST
I apologize, I submitted the bug report before I was finished writing it. The text under "Additional info" was a note to myself. Here is the real thing:

Additional info:

After hacking at it for half a day I couldn't figure out a simple reproduction of the error I described using just hivexsh. I also couldn't reproduce the error with a fresh user hive (from a newly created user). That leads me to suspect that it may have something to do with the way the contents of a changed node are appended to the end of the file, and that that may have issues once the hive reaches a certain size.

I also wanted to refer to some interesting information I found in the ntpasswd project: Apparently the hive format has been extended once again, and a document detailing the hive format has helped the ntpasswd to achieve excellent support up to hives from Windows 8.1. See http://pogostick.net/~pnh/ntpasswd/editor.html and http://pogostick.net/~pnh/ntpasswd/WinReg.txt .
Comment 4 Richard W.M. Jones 2016-01-07 11:36:00 EST
This Python code makes my head hurt.

Can you narrow it down to a single change or set of changes that
causes the problem?

Does the Python code preserve the registry node types when making

One thing I did notice is that the code truncates the final (UTF-16LE) \0
from all the strings that it modifies, and this is the sort of thing
which Windows might not like.  Try doing:

hivexregedit --export NTUSER.DAT.orig '\\' > NTUSER.DAT.orig.reg
hivexregedit --export NTUSER.DAT.result '\\' > NTUSER.DAT.result.reg

and diff the two files, and you will see what I mean.
Comment 5 Sebastian J. Bronner 2016-01-14 13:25:55 EST
Your observation about the \0 terminators took care of my problem (see below). Thanks!

After having tried to narrow down the problem for the better part of a day, I had given up and decided to submit the complex reproduction that you saw. However, as you already found the culprit, the problem thankfully no longer needs further narrowing.

Yes, the node type is preserved by my python code when making changes (line 92).

The truncated \0 was definitely a problem. The value_string function returns a string without the \0 terminator. After changing that string and passing it to the node_set_values function that terminator was missing. I changed my code to append the \0 before passing it to node_set_values. That solved the issue I was seeing.

I would like to suggest, however, that some adjustment of the documentation and/or the library code might help alleviate confusion on this point. I wasn't expecting to have to change the string returned by value_string before passing it to node_set_values, except encoding it to utf16-le, as that is mentioned in the documentation. I was reading http://libguestfs.org/hivex.3.html as well as the python binding source file.

My suggestion is this: Either (a) the node_set_function could be changed to add the 0 when missing for string types, (b) the need to add a terminator could be added to the documentation and/or (c) the value_string could return strings with the \0 terminator.

Many thanks for your help. My code is now productive.

Note You need to log in before you can comment on or make changes to this bug.