Bug 1296595

Summary: libhivex: use on Windows 7 NTUSER.DAT causes damage that manifests itself with folder rename errors
Product: [Community] Virtualization Tools Reporter: Sebastian J. Bronner <waschtl>
Component: hivexAssignee: Richard W.M. Jones <rjones>
Status: NEW --- QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: ptoscano, rjones, waschtl
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
working user hive before editing with libhivex
none
python program using libhivex, required for the testcase
none
user hive resulting from edit that causes folder rename errors none

Description Sebastian J. Bronner 2016-01-07 15:55:49 UTC
Created attachment 1112486 [details]
working user hive before editing with libhivex

Description of problem:
=======================

After editing an NTUSER.DAT hive, taken from a Windows 7 user account, with libhivex, that user receives an error when renaming folders.

Version-Release number of selected component:
=============================================

python3-hivex 1.3.11
libhivex0 1.3.11

How reproducible:
=================

always for large hives (a few megabytes)

Steps to Reproduce:
===================

1. Take the attached hive NTUSER.DAT.orig and python program registry-replace.

2. Run the following command:
   registry-replace -p '\\sbr' -r '\\sbr' NTUSER.DAT

3. The attached hive NTUSER.DAT.result is the result.

4. Install it in a Windows 7 user profile named sbr.

5. Log in as that user.

6. Rename a folder (or create a new folder specifying a name). The location doesn't seem to matter, but the My Documents folder is where I tested it.

Actual results:
===============

The registry hive is successfully altered, the new values are visible in the user account. However, the user always faces the following message when renaming a folder (also applies when creating a new folder with a non-default name):

"""
Rename Folder

An unexpected error is keeping you from renaming the folder. If you continue to receive this error, you can use the error code to search for help with this problem.

Error 0x80004005: Unknown Error

[Repeat] [Cancel]
"""

After clicking on [Repeat] the folder is renamed (or created) successfully.

Expected results:
=================

In addition to the hive being successfully altered, renaming (and creating) folders should continue to be possible without error messages popping up.

Additional info:
================

The library soandso manages to do this. Therefore, I am assuming it has something to do with some new way that hives are organized in Windows 7 and later. ...

Comment 1 Sebastian J. Bronner 2016-01-07 15:57:28 UTC
Created attachment 1112487 [details]
python program using libhivex, required for the testcase

Comment 2 Sebastian J. Bronner 2016-01-07 15:58:28 UTC
Created attachment 1112488 [details]
user hive resulting from edit that causes folder rename errors

Comment 3 Sebastian J. Bronner 2016-01-07 16:12:07 UTC
I apologize, I submitted the bug report before I was finished writing it. The text under "Additional info" was a note to myself. Here is the real thing:

Additional info:
================

After hacking at it for half a day I couldn't figure out a simple reproduction of the error I described using just hivexsh. I also couldn't reproduce the error with a fresh user hive (from a newly created user). That leads me to suspect that it may have something to do with the way the contents of a changed node are appended to the end of the file, and that that may have issues once the hive reaches a certain size.

I also wanted to refer to some interesting information I found in the ntpasswd project: Apparently the hive format has been extended once again, and a document detailing the hive format has helped the ntpasswd to achieve excellent support up to hives from Windows 8.1. See http://pogostick.net/~pnh/ntpasswd/editor.html and http://pogostick.net/~pnh/ntpasswd/WinReg.txt .

Comment 4 Richard W.M. Jones 2016-01-07 16:36:00 UTC
This Python code makes my head hurt.

Can you narrow it down to a single change or set of changes that
causes the problem?

Does the Python code preserve the registry node types when making
changes?

One thing I did notice is that the code truncates the final (UTF-16LE) \0
from all the strings that it modifies, and this is the sort of thing
which Windows might not like.  Try doing:

hivexregedit --export NTUSER.DAT.orig '\\' > NTUSER.DAT.orig.reg
hivexregedit --export NTUSER.DAT.result '\\' > NTUSER.DAT.result.reg

and diff the two files, and you will see what I mean.

Comment 5 Sebastian J. Bronner 2016-01-14 18:25:55 UTC
Your observation about the \0 terminators took care of my problem (see below). Thanks!

After having tried to narrow down the problem for the better part of a day, I had given up and decided to submit the complex reproduction that you saw. However, as you already found the culprit, the problem thankfully no longer needs further narrowing.

Yes, the node type is preserved by my python code when making changes (line 92).

The truncated \0 was definitely a problem. The value_string function returns a string without the \0 terminator. After changing that string and passing it to the node_set_values function that terminator was missing. I changed my code to append the \0 before passing it to node_set_values. That solved the issue I was seeing.

I would like to suggest, however, that some adjustment of the documentation and/or the library code might help alleviate confusion on this point. I wasn't expecting to have to change the string returned by value_string before passing it to node_set_values, except encoding it to utf16-le, as that is mentioned in the documentation. I was reading http://libguestfs.org/hivex.3.html as well as the python binding source file.

My suggestion is this: Either (a) the node_set_function could be changed to add the 0 when missing for string types, (b) the need to add a terminator could be added to the documentation and/or (c) the value_string could return strings with the \0 terminator.

Many thanks for your help. My code is now productive.