I dont think it is essential. I compiled small list years ago on my A1200 running some MorphOS 0.x beta.

To make it useful it shuold have more information than just works/doesnt work:

* an application name and version (it must be possible have multiple versions for same application)
* OS version it was tested on
* when it was tested
* who tested
* what is needed to make it work
* an explanation (if possible) when it does not work
* application category (to find usable software)
* user commentary

But I feel many users wouldnt use it...