Bug 1025106
| Summary: | [XMLRPC][Serialization] Improve model's serialization | ||
|---|---|---|---|
| Product: | [Other] TCMS | Reporter: | cqi |
| Component: | XMLRPC | Assignee: | Yang Ren <ryang> |
| Status: | VERIFIED --- | QA Contact: | |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | Devel | CC: | ryang |
| Target Milestone: | --- | Keywords: | Improvement |
| Target Release: | 3.8.6 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Enhancement | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | Type: | Bug | |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Yeah, this sounds doable. So, instead of getting each field value by traversing all the fields of a model, a better way would be to calculate a list of Django LookUp field names and give it to the .values() method. Okay, I'd like to see the how much of a difference it makes for an existing XMLRPC API. Also, I don't suspect this introduces inconsistency into XMLRPC APIs right? (In reply to Chaobin Tang from comment #1) > Yeah, this sounds doable. > > So, instead of getting each field value by traversing all the fields of a > model, a better way would be to calculate a list of Django LookUp field > names and give it to the .values() method. > > Okay, I'd like to see the how much of a difference it makes for an existing > XMLRPC API. > > Also, I don't suspect this introduces inconsistency into XMLRPC APIs right? Yes. Rewrite must ensure the original behavior not changed. Cache mechanism could be introduced to cache each model's fields' names. Because, they are immutable during the runtime once TCMS begins to run. |
Model's serialization is implemented in module ``tcms.core.utils.xmlrpc``. An object's all fields are serialized into a dict object recursively, which means all field with type ForeignKey and ManyToManyKey will also be serialized as well by getting related object. However, only the name or title, whatever a description text, of the related object is required besides ID. Current implementation has two major problems introduced by the ForeignKey field. - Each time to get related objects following the relationship represented by a ForeignKey field, a SQL query happens, and a complete object with all fields will be generated, although only the two field ID and name or title are required. Thus, this can produce amount of SQL queries if a queryset is being serialized. - Further, too much unnecessary data is loaded from MySQL and be transferred over network from MySQL to TCMS. From above two point, a proposal solution is a new serialization method that operates upon a explicit fields list with names rather than the field objects of a Model object. For example, the new method would operate on a QuerySet, ``TestRun.objects.filter(**query).values('run_id', 'summary', 'start_date', 'plan', 'plan__name', 'tag__pk' ...)`` the fields list here can be generated programmatically from a Model. This solution can covers both above problems. In this way, all necessary data is retrieved via one SQL query. Again, to serialize a TestRun object, only the id and name of related TestPlan object are required. It's worth to consider to make an improvement to this core method of XMLRPC API. Model's change would bring less affect to it only.