Market research for requirements analysis using linguistic tools

Numerous studies in recent months have proposed the use of linguistic instruments to support requirements analysis. There are two main reasons for this: (i) the progress made in natural language processing and (ii) the need to provide the developers of software systems with support in the early phases of requirements definition and conceptual modelling. This paper presents the results of an online market research intended (a) to assess the economic advantages of developing a CASE (computer-aided software engineering) tool that integrates linguistic analysis techniques for documents written in natural language, and (b) to verify the existence of the potential demand for such a tool. The research included a study of the language – ranging from completely natural to highly restricted – used in documents available for requirements analysis, an important factor given that on a technological level there is a trade-off between the language used and the performance of the linguistic instruments. To determine the potential demand for such tool, some of the survey questions dealt with the adoption of development methodologies and consequently with models and support tools; other questions referred to activities deemed critical by the companies involved. Through statistical correspondence analysis of the responses, we were able to outline two “profiles” of companies that correspond to two potential market niches, which are characterised by their very different approach to software development.

1. Multi-year project funded by the Department of Computer and Management Sciences of Trento University.

2. Some comparisons deriving from our research are described in [1].

3. For further study of issues related to online market research, the interested reader can refer to the literature (see for example, the publications found at ESOMAR – European Society for Opinion and Marketing Research – http://www.esomar.org).

4. A bibliography is available at http://nl-oops.cs.unitn.it.

5. The first proposals to use linguistic criteria for the extraction of entities and relations, and then objects and associations, from narrative descriptions of requirements date from the 1980s [7].

6. Included in this category are, for example, the instruments described in [10] and [11].

7. For example, to recognise if Washington is the name of a person, of an airport, or of a city in a given document requires a semantic approach. Limitations on space do not permit a deeper discussion of this issue here; see for example [12].

8. “The hard part, and the true essence of requirements, is trying to understand your customer’s needs. A person involved in requirements needs human skills, communication skills, understanding skills, feeling skills, listening skills” [13]. See also [14].

9. For a recent study on why it is impossible for users to know their requirements beforehand, see [16].

10. On this point, see, for example, the tasks required by the MUC competitions (Message Understanding Competition) organised by the DARPA (Defense Advanced Research Projects Agency) [17].

11. The official documents of the UML’s specifications can be found on the OMG (Object Management Group) website: http://www.omg.org.

12. Natural Language – Object-Oriented Production System, http://nl-oops.cs.unitn.it.

13. The questionnaire is available along with the data gathered and other related research material at http://on-line.cs.unitn.it.

14. For example, a questionnaire like the one used for the survey described in [23] would have to be radically altered to be used online.

15. In light of the observations in [24], this may not be so surprising.

16. The choice of tools for question 14 was made on the basis of sales data for a period prior to the study.

17. Because the survey concluded at the end of the Arena opera season, the tickets were replaced with CDs of opera music by Verdi.

18. One of the aims of the survey, in fact, was to investigate the conditions under which newsgroups can be used to carry out online surveys.

19. Limited number of questionnaires obtained (44) and accusations of spamming.

20. This is a rather high percentage, bearing in mind that they were collected from the homepages of official company websites. Another survey carried out in the same period on winter tourism, where the addresses were provided by a specialized magazine, found a very similar percentage of wrong addresses (8.9%), but the amount can be much higher. For example, in a survey of Internet users carried out in 1996, 35% of a total 1221 addresses were found to be wrong [26].

21. This was the minimum value for the traditional-type surveys, which achieved a maximum response rate of 20%. In the survey described by Glass and Howard [25], the percentage rose to 17% after the questionnaire mailings were supplemented by telephone contacts with fax follow-up.

22. For a survey on virtual supermarkets, a message was sent to 6 newsgroups obtaining 100 completed questionnaires.

23. All the percentages were calculated on the total number of respondents who answered the relative questions, with non-replies omitted.

24. Further investigation of this aspect would require knowledge of the number and size of the companies’ customers. This, however, is beyond the scope of our survey.

25. For an introduction to the evaluation of potential demand, see for example, [27].

26. In the past, the need to support different graphic notations was a drawback to the market for CASE, in that it required producers to choose which notation to support with their own tools, or to absorb the higher cost of developing different versions.

27. A CASE based on linguistic techniques for object-oriented analysis does not necessarily require the realisation of an entire support environment, but rather can be seen as a module that can be integrated with an existing product.

28. A study of the ‘robustness’ is of utmost importance also to establish the degree of analyst intervention required in developing requirements models, and should be conducted using a prototype of the tool. See also point (b) of the introduction and conclusion.

29. None of the tools indicated by those choosing the option ‘Other’ was selected more than twice.

30. International Data Corporation (IDC) data.

31. These figures seem to contradict the results of the survey by Glass and Howard [25], where CASE technologies are described as being in decline. However, it should be pointed out that where back-end or ‘lower’ CASE technologies are concerned, many of the functions offered by these tools are by now part of the development environment. Moreover, other expressions are often used instead of ‘CASE’: for example, the IDC surveys use OOAMDC (object-oriented analysis, modelling, design, and construction) tools. On the other hand, in 1998 the market for OOAMDC grew by more than 10% (24% in Europe). See also the results in [29].

32. It should be pointed out, however, that the data of our survey are expressed in terms of units of output by the companies surveyed, while the sales figures are calculated on invoices and consequently depend on the prices charged by vendors.

33. To be noted is that also around one-third of the final observations concerned the role and importance of requirements. Taking into account the different goals of the surveys described in [30, 31], we can compare these results with those obtained for a question therein on the perceived relative importance of software problems in Europe (most of the software problems are in the area of requirements specification and managing customer requirements; following documentation and testing) and on the perceived scope of a generic process model (defining system requirements, 78%).

34. In this regard we quote a remark made in one of the questionnaires: “I hate to be a cynic, but there are hardly any worthwhile tools. The overhead in learning to use them is too great for the payoff.”

35. The contingency table is available at http://on-line.cs.unitn.it.

36. Notable exceptions are the surveys conducted by the European Software Institute: http://www.esi.es.

37. These surveys were carried out with different objectives and using different methods and samples. The survey described in [25] used 78 questionnaires compiled mainly by directors or managers of information systems development in companies operating outside the software field, while the Finnish one reports results relative to 12 Finnish companies, 8 of which worked exclusively in the software field.

Authors and Affiliations

1. Department of Computer and Telecommunication Technology, University of Trento, Via Sommarive 14, 38050, Trento, Italy

   Mich Luisa

2. Department of Computer and Management Sciences, University of Trento, Via Inama 5, 38100, Trento, Italy

   Franch Mariangela & Novi Inverardi Pierluigi

Corresponding author

Correspondence to Mich Luisa.

An erratum to this article can be found at http://dx.doi.org/10.1007/s00766-004-0195-3

Appendix A:

1.1 Questionnaire for a new CASE tool

Appendix B:

1.1 Online material

(http://online.cs.unitn.it/)

• Questionnaire (html form)

• Contacted newsgroups list

• E-mail messages

• Correspondence analysis

Reprints and permissions