5

I am seeking pointers to assist the direction of my research into providing assessements of functionality, usability or elegance in system development.

Can you provide references to work being done on the measurement of functionality, usability, or coding style? Who (person/organisation) is doing work in this area? Where can I find such references.

I have developed some ideas about system construction, that are a little different to mainstream development. The starting point is a an adequate description of the problem. I am developing a demonstration/proof of concept project.

In developing my proof of concept project I have found an unexpected and unlooked for benefit. So far, I have found that useful information about the completeness of a system can be gathered by taking an indirect view of the development. This indirect view is based on the description of the problem, rather than the software solution.

I have also become convinced as a result of these developments that it may well be possible to provide guidance from my approach to system development in other areas such as guidance on how functional the system is; how usable the system is; or how elegant the solution is.

My explorations so far have led to the following suggestions, and my responses to them:

  1. Literate programming is elegant. - Literate programming may well be elegant, but this shifts the problem back - how do you assess how literate a program is?
  2. That I am looking for the system development equivalent of a measure of aesthetics ie the ineffable - while I appreciate that such is not possible, I still believe it possible to give guidelines from the infomation available during the course of developing a system.
  3. That what I am seeking can only be assessed after a solution has been in use for some time, and only by comparison with other solutions to the same problem. - This may well be the case, and may indeed the rock on which my search founders. However I still believe that it is possible that measures from the development may throw some heuristic insight on these aspects of software.
  4. Function point analysis is a measure of functionality - I see FPA as more a measure of productivity than functionality. It does not tell you how much functionality has been incorporated into the system, bur rather how much has been included compared to a baseline derived from the same context. As the context is continually evolving, this detracts from the measure's usefulness.
  5. That these concepts cannot be measured and there is no agreement on what constitutes functionality, usability, or elegance and there can't be for similar reasons to the argument - I am stubborn enough to believe I can, at least partially, refute this suggestion by providing some assistance to system developers.
  6. That the information I am looking for lies outside of system development; in the graphic arts arena; psychology; biology; or other - this is looking more and more likely.
  7. Utilising the objects of the system directly as its user interface - this shows promise for a class of systems designed to facilitate problem solving.
  8. The lessons from traditional graphic art (layout of complex data) can be transferred to system development - this is looking like the most promising route and I am trying to establish correspondance with a leading graphic designer. This may only be of use for informational systems but does look as if it is much more wide ranging than that.
  9. It has also been suggested that I am being an "architecture astronaut", out of touch with reality - this may be the case, but if it is so, then I am likely to be the last to realise it and such a prospect does not deter me from my search.
alain.janinm
  • 19,951
  • 10
  • 65
  • 112
Chris Walton
  • 2,513
  • 3
  • 25
  • 39
  • So literate programming isn't elegant? http://en.wikipedia.org/wiki/Literate_programming – James Black Sep 11 '10 at 15:32
  • 5
    I've been looking for a quantitative measure of the artistic merit of a photograph... by which I mean you are seeking the ineffable. If elegance were measurable, myriad, endless university level BS sessions would cease to exist (and that would be sad). – msw Sep 11 '10 at 15:33
  • @James - I did not mean to imply that literate programming was not elegant - I did suggest that the definition I gave was initial, partial, and derived from mathematics. Literate programming certainly can be elegant, but does not seem to be directly correlated to elegance - a program can be any combination of (non-)elegant and (non-)literate. – Chris Walton Sep 11 '10 at 15:40
  • 1
    Quite a few candidate definitions are offered in previous question http://stackoverflow.com/questions/563036/what-is-elegant-code. I personally like this definition from Antoine De Saint Exupery, the author of "The Little Prince": "Perfection is attained not when there is no longer anything to add, but when there is no longer anything to take away..." – Andy Thomas Sep 12 '10 at 11:54
  • Thanks @Andy - the list of points from gw in that previous answer is exactly what I am looking for, though some of those points I would have included in other categories (non-functional requirements?). – Chris Walton Sep 12 '10 at 17:39
  • Progress Report I have been investigating work being done by graphic layout designers (printed presentation of complex data), and ergonomists. This is throwing up some useful fragments of information. – Chris Walton Oct 19 '10 at 16:23

4 Answers4

1

I think some of these features would have to be measured after the product has been in use for quite some time, as compared to another, similar implementation.

Consider the example of multiple GUI implementations for a piece of software. You could measure things like how long it took a user to accomplish a certain task using one specific implementation relative to accomplishing the same task (for the nth time) on the (almost) same piece of software using different GUI implementation. This would provide for some sort of a useful metric in relative terms.

Going along this path might help you clarify these ideas in terms of getting (publishable?) results. From reading your original description, it sounds like you are looking for absolute rather than relative metrics. However, from quickly looking over this question and trying to come up with an interesting, useful, reply during a five minute break from my work, the relative metric was the best I could come up with.

I hope this helps,

Brian J. Stinar

Brian Stinar
  • 1,080
  • 1
  • 14
  • 32
  • thank you. Your point on duration as a factor in assessing elegance is well taken. Absolute metrics would be nice, but I do appreciate that metrics for many of these points can only be relative. This was the reason for my reference to heuristics originally. – Chris Walton Sep 13 '10 at 15:05
1

Of the three things you list, usability is the most measurable. Doing a search for "measuring code usability" will yield many hits, for everything from websites to parallel programming.

Some highlights:

ISO standards on software engineering; Here you'll find the standards for product quality and the software development lifecycle:
http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_tc_browse.htm?commid=45086

The Cliff Notes version of the ISO standards : )
http://www.usabilitynet.org/tools/r_international.htm

Standards for software ergonomics:
http://www.iso.org/iso/catalogue_detail.htm?csnumber=52712


From a well-written paper on parallel program usability, found here:

Several features of a PPS determine its usability. Among these are:
1) Learning curve: How long does it take an expert or an inexperienced parallel programmer to be able to use the PPS productively? Note that some PPSs specifically address the needs of experts, while others are targeted at novices; few are suitable for both.
2) Programming errors: Some systems restrict the use of parallelism to prevent errors (e.g. Enterprise). Other systems, such as NMP and PVM, allow the user to do anything, trading flexibility for a higher chance of programming errors. Usually the potential for errors is directly related to the number of lines of user code. Therefore, systems that require more user code may be more susceptible to errors.
3) Deterministic performance: Non-determinism, common in the implementation of some algorithms and inherent in some PPSs, can significantly increase the overhead in application debugging.
4) Compatibility with existing software: Legacy software cannot be ignored. Ideally, the PPS must support the integration of existing software with minimal effort.
5) Integration with other tools: A PPS should either come with, or provide access to, a complete suite of software development tools including facilities for debugging, monitoring and performance evaluation.

An article on quantifying and measuring functionality:
http://www.computer.org/portal/web/csdl/doi/10.1109/METRIC.1999.809732

Links to the CUE-4 Hotel Pennsylvania usability studies, where 17 independent teams performed usability of the website for Hotel Pennsylvania
http://www.dialogdesign.dk/CUE-4.htm

This Wikipedia article has a lot of links to software-quality-related articles. The article itself discusses a number of focal points for software quality, including Understandability, Conciseness, Consistency, Maintainability, Testability, Usability, Reliability, and Efficiency, among others.

http://www.drdobbs.com/windows/184405654;jsessionid=SB2LUABORKQHBQE1GHOSKHWATMY32JVN
The author discusses techniques that Microsoft uses to design and evaluate the usability of their APIs.

Another suggestion: Go to some of the more renowned colleges for software engineering and nose around on their Computer Science homepage for published articles on the subject.

As others have said, establishing quantitative measurements on these principles is like nailing jello to a tree... but I disagree that they can't or haven't been studied in quantifiable analyses.

HTH!
James

James King
  • 6,233
  • 5
  • 42
  • 63
0

All of those concepts cannot be measured. They can't even be objectively agreed upon.

I dare to say they have no clear interpretation in the physical world. They only exist in human mind. Every other person will perceive and measure those in accordance with their life experience, knowledge, experience and attitude to the problem field, development of engineering, artistic and interpersonal abilities. Evey if you can force some individuals to "measure" it, it will be highly subjective.

How do you define beauty and love, joy and sadness? Usability and efficiency will have much to do with those.

Some ideas can come from psychological studies. But only a few ideas. At best you can apply this knowledge to try to evoke some specific response in the users' minds. But it might work or it might not.

When you can't understand precisely the model of response of the users, you can't plan for specific response. Consequently, you can't measure the degree of a particular characteristic of your program. So you can't define the scale in the first place.

  • I may be Quixotic, tilting at imaginary giants, but I am trying to make a start on defining the scale. An example of my approach is not mentioned in the question - completeness. I take it as given that assessment of completeness is impossible - eventually Godel's Incompleteness Theorum will bite one in the bum. However, by changing one's point of view - by looking not at measurement of solution completeness, but looking at measurements arising from the description of the problem, one can say a great deal about the completeness of the solution. – Chris Walton Sep 13 '10 at 17:09
0

I am answering my own question to give an indication of how far I have got in obtaining an answer to my original question. It is apparent that any metrication of concepts related to elegance will share the following characteristics:

  1. they are likely to be heuristic
  2. they are likely to have to be based on multiple iterations of the development process
  3. they are likely to be relative rather than absolute
  4. they are likely to be based on areas of study other than software/system development, in addition to needing information from the system development itself. Examples of possible domains are aesthetics, psychology, neurolinguistics and neuroaesthetics;
  5. If the work I am doing is to be of any value to software developers, it must not be based on complex, mathematical or statistical models, but must offer guidelines based on their work, and work by other developers.
  6. Any guidelines are likely to be based on the answers to questions like:
    • What is the frequency of objects, attributes, data types?
    • How does this frequency compare with other projects?
    • Is this measure of any value in aiding the developer to evaluate his project/see room for improvement?

I am still seeking references to work being done in this arena.

Donal Fellows
  • 133,037
  • 18
  • 149
  • 215
Chris Walton
  • 2,513
  • 3
  • 25
  • 39