The following framework comparison exercise was reported in June 2022 and at least one publication covers the comparison methodologies used:
Regarding the terminology in this context:
- a framework essentially refers the codebase and supporting software development infrastructure
- a model essentially refers to a particular instance on which a number of scenarios are run in order to investigate some predefined research question
Some care is required to distinguish between framework quality and model quality therefore — although clearly the two are intertwined. There is a considerable literature on software quality assurance and practice, but less so on model quality assurance.
Good software engineering suggests that the programming should not contain hardcoded parameters and that all necessary values are read in at runtime. Model quality is critically dependent on data availability and so data quality assurance necessarily plays a key role. Model quality is also contextual on the purpose of the research being undertaken.
The quality of production data under statutory reporting in Europe is poor as the posting below indicates. And some market information under statutory reporting in Europe is actively made more difficult to recover while still complying technically with the requiring legislation (doubtless in an attempt to increase data sales revenues). Moreover, neither class of information is served under clear legal terms concerning its use and reuse.