Description of problem: During startup there's deserialization exception. See bug #1372950. Analysis: probably during org.ovirt.engine.core.bll.tasks.CommandExecutor#executeAsyncCommand command parameters are serialized to json and stored into db. Parameters classes are not prepared to handle serialization into json, so they makes no effort making that simple and working (and they shouldn't do that). But utils performing such serialization makes also no effort finding that out, which ends in situation, where immutable is being serialized into db. Deserialization of such data fails, because immutable objects does not have default constructor. Actual results: Expected results: as we cannot get command parameters classes to cooperation CommandEntityDaoImpl needs to altered scanning for immutable objects, in our case namely collections. Such collections should be either fixed or command rejected as is. Also(!) whole engine halts because of once un-deserializable command/some tasks?? I'd consider that as a far bigger problem that everything above. Additional info:
We should also consider to avoid storing serialized things in the database. It is nearly impossible to guarantee that those things can be de-serialized after an upgrade.
well — *for sure* — that's true. However many our features (some of them patented) are dependent on usage of this anti-pattern. So there isn't a way (at least not an easy one) how to avoid this. But if you use jackson with default configuration, then assuming you're serializing some 'libraries' as I do in my example, you end up with: {"name":null,"shelves":[{"name":null,"books":[{"name":"L'Écume des jours"}]},{"name":null,"books":[{"name":"Brave new world"}]}]} which, as you realize, does not suffer from any deserialization issues - at all. You need to manually configure json serializer, to include types, using: mapper.enableDefaultTyping() to get: {"name":null,"shelves":["java.util.ArrayList",[{"name":null,"books":["java.util.ArrayList",[{"name":"L'Écume des jours"}]]},{"name":null,"books":["java.util.Collections$SingletonSet",[{"name":"Brave new world"}]]}]]} which cannot be deserialized. (As a little surprise for me is that singletonList is serialized as ArrayList, but that's not that important now). And we do just that in org.ovirt.engine.core.utils.serialization.json.JsonObjectSerializer (nicely in static initializer). We're actively asking jacksons, to produce output, which might be one, which we won't be able to deserialize. But if we drop that line, we might have other problems with deserializing certain parameters classes(where there are interfaces used).
Martin - please consider the options here, and have someone working on that during 4.1.
adding CodeChange keyword. Deserialization issue was happening in coco framework and not in engine, while reason for coco failure was in engine code. To verify it, specific command would have to be invoked, but such command would have to be altered so that it fail leaving db in invalid state for next engine startup. While this is actually testable, it's probably not that easy to do so, and change isn't marginal flow so it's sufficiently proven by unit tests and everyday work of developers/testers.
The fix for this issue should be included in oVirt 4.1.0 beta 1 released on December 1st. If not included please move back to modified.
ok, based on #4