In terms of compliance of this software with ISO standards or other modelling software standards and computer software modelling methods. Can you please indicate what compliance standards are used for the development of oemof? or what are the quality assurance indicators used for developing and maintaining the software modelling processes
Hi @Jessica My reply concerns standards compliance more generally and not just in relation to the oemof project. I am not aware of any discussion on ISO software engineering or business systems standards being applied to open source software development in any domain. Quite probably because the two exist within different universes. (That said, patent-encumbered technical standards — also known as standard essential patents — are widely debated within the open source world.) Did you have a particular ISO standard in mind?
Our community has traversed the more general issues of quality control and validation for open energy system models. You might find some useful material there?
My favorite article on real world quality control in relation to open source software development in a research context is Crusoe and Brown (2016). Keeping up with emerging technical (as opposed to software engineering) standards, whether formalized or not, proved beyond their reach in relation to their resourcing.
Somers (2017) discusses the problem of quality assurance in relation to mission critical software. Energy modeling is not mission critical but that is no excuse for sloppy programming.
References
Crusoe, Michael R and C Titus Brown (29 November 2016). “Walking the talk: adopting and adapting sustainable scientific software development processes in a small biology lab”. Journal of Open Research Software. 4 (1): e44. ISSN 2049-9647. doi:10.5334/jors.35.
Somers, James (26 September 2017). “The coming software apocalypse”. The Atlantic. Washington DC, USA. ISSN 1072-7825.
Hi @robbie.morrison and thank you for the prompt response,
My question concerns ISO/IEC 12207, ISO/IEC 9126-1 and ISO/IEC 29119 for software development, software quality and software testing.
My understanding is that the listed standards relate to the development and publishing of any software.
ISO/IEC 12207 governs the technical management processes and the technical processes of software development like:
software requirements, system analysis, data verification, transition, data validation, operation and maintenance, QA and measurement etc…
The quality control and validation processes for open energy system models listed under the links are interesting especially the code correctness and data accuracy parts.
The article/paper you cited covers very interesting aspects and processes of these standards, it also refers to the checklist (in the following link: checklist ) as basis for rational development software process in page 2 of the paper:
“To guide our development of a rational software development process, we applied the Software Sustainability Institute’s Criteria-based assessment checklist [7]”
This checklist was developed based on the ISO/IEC 9126-1.
Therefore my question becomes whether this checklist or the guidelines listed in the research document being cited are followed during the software development/publishing/maintenance or otherwise what other development procedures for quality and testing are followed for the development and maintenance of oemof ? and whether any corresponding certification was granted.
Regards
Jessica
Hello @Jessica In all my dealing with free and open source software (FOSS) over the years, I have never heard the ISO/IEC standards you mention discussed once. One reason could be that these documents are prohibitively expensive. But a more likely explanation is that FOSS developers believe that open development naturally produces quality software without resorting to high levels of formalism: that public code and many and diverse developers will naturally spot and fix bugs. That contention has been debated at length and I won’t repeat the arguments here, you can search the web instead.
But what I will say is that the open energy modeling community has been steadily embracing software engineering principles. When I started with open energy system models in 2003, complex code (15k source lines of C++) was not even under version control. Nowadays, most projects (oemof included) run on GitHub, use issue trackers, and seek and encourage developer and user participation. Some have adopted coding guidelines as well. I actually wonder how much emphasis the ISO/IEC standards you quote place on community building and FOSS development methods more generally? HTH, R.
PS: as an administrator, I changed my first name to my @‑handle in your post. We encourage the use of such handles because if we need to remove an account as a result of a data privacy request, the forum software can automatically anonymize the relevant handles.
References
Jackson, Mike, Steve Crouch, and Rob Baxter (ongoing). Software evaluation guide. Software Sustainability Institute (SSI), University of Edinburgh. Edinburgh, Scotland. Website.
Spinellis, Diomidis (3 April 2006). Code quality: the open source perspective. Upper Saddle River, New Jersey, USA: Addison-Wesley Professional. ISBN 978-0-321-16607-4.
New paper on open model maturity, although standards compliance is not traversed.
References
Groissböck, Markus (1 March 2019). “Are open source energy system optimization tools mature enough for serious use?”. Renewable and Sustainable Energy Reviews. 102: 234–248. ISSN 1364-0321. doi:10.1016/j.rser.2018.11.020. Closed access.