Some thoughts on the provision of future problems

It is always interesting to see the progress that new members make when they first discover the site. You can feel their enthusiasm as they rapidly add to the number of problems solved. Unfortunately I think that too many of them soon reach a point at which they decide that the problems are getting too difficult. They then lose interest and stop coming back to the site. This is a real shame because of all that the site has to offer.

I am sure that many more people would regularly visit the site if they had an expectation of new problems which they would have a good chance of solving. I recognise the conflict between the addition of new problems and the preservation of standards with the two certificates on offer. I think that the certificate system is one of the best elements of the site and would not want to see the certificates degraded in any way. However, there are obvious ways of adding new problems of varying standards without any degradation of the certificates.

One method of achieving this would be to state that only the first 240 problems can have their solutions count towards a certificate. I have chosen 240 as this is the current number of problems but any reasonable choice of the maximum problem number would do instead.

Alternatively, new problems could be categorised such that problems not reaching the required difficulty level for a certificate could be clearly marked as such. However, this method is less straightforward than the previous approach. I am sure that there are probably several other compromises which could be considered as alternatives.

If people are encouraged to keep on returning to the site because of the likelihood of amenable new problems, they are probably also more likely to try some of the existing problems that they haven’t yet been able to do. The issue of encouraging people to return to the site applies right through the difficulty levels. Even for those who have achieved certificate level it can be disheartening to think that there are no new problems coming along that are within their grasp. The most recent 10 problems have just 64 solutions between them, an average of 6 or 7 per problem and, not surprisingly, it is the same small group of solvers for most of them. The most challenging problems are great and I am sure that they are really popular with the small group of members who can solve them. I just think that it would be so much better if there was a good mix of difficulty levels within the new problems. This would bring more people back to check them out, which in turn would lead to more people tackling some of the other problems.

It might also help to have a more convenient way for people to check on new problems. One possibility is to have a new menu option for the most recent 10 problems. These would be listed in order of newness with a small amount of useful additional information. The current number of solvers, together with a very brief introduction would probably be appropriate. The most important aspect of the introduction would be to give an indication of the difficulty level of the problem. This is something that is not easy to judge but some indication is more helpful than none. If some particular skill or area of knowledge is needed then that also could be included here. I don’t think that information like this gives too much away but it would help someone to judge whether or not they are likely to be able to do the problem. It might also encourage them to explore the area of specialist knowledge deemed appropriate for the problem. I realise that some of this is information is already provided in the problem description but I think a most recent 10 problems summary could be quite useful.

Next there is the issue of providing a succession of new problems. It certainly seems to me that this is far too much for Rodion to do without any help; although he seems to have done a remarkable job so far. It is clear that a number of the current members (myself included) are happy to provide fairly detailed ideas for new problems. I wonder if it would be worth having a new forum section specifically for ideas for new problems. Members could put their suggestions here (Ideally with some detail) and this can then be commented on and possibly refined by other members through the forum. Ultimately the accepted suggestions would get the appropriate makeover in Rodion’s individual style.

Rodion, thanks for the interesting and detailed response.

If a cut-off of the first N problems is used for the certificate then one possibility would be to set N at the number of problems that existed when the present certificates were introduced. Some notification of this change would be needed for people who are currently close to completion but this method would be a simple way of retaining the original difficulty level associated with the certificates.

If you move to new certificates based on a list of designated problems then you have the definite advantage of ensuring that only suitable problems are used. I agree with the removal of the very easy ones and others that might encourage cheating. I assume that you would require only a fixed number of these designated problems to be solved in order to gain the certificate. That would include a reasonable level of choice for the person solving them. However, if the problems are in categories, some additional restrictions can be imposed such as “problems must be solved from at least X categories”. Even with this system you have the difficulty associated with adding new designated problems. This could be done by setting the target number of problems to be a fixed proportion of the designated problems rather than a fixed number of them. Alternatively, if the designated problems are chosen with sufficient care there would be no need to add to them. All new problems would not count towards certificates.

With even a small number of members interested in suggesting new problems it should be possible to provide them with sufficient frequency to keep people coming back to the site. Ideally a rate of one new problem every two weeks would be a good target to aim for.

I think that the existing forum structure (with a dedicated section) is a good starting point for a discussion of new problem ideas. This makes it easy for any of the members to join in. There is no need for privacy because the details being discussed are likely to be the same details which would eventually appear in the final problem description. The discussions can easily include sample data and results so that those taking part can try out the ideas if they wish to do so. They would still need to solve the problem with new data at a later stage if it becomes accepted. The only need for privacy comes in when there is a need to produce problem checkers or generators. These would only be needed at a late stage in problem development and e-mail should suffice for this purpose.

I agree with you on the point of not creating any more new ranks, at least not for some time. There is certainly no point in adding a new rank after every 20 new problems. A better yardstick might be to wait until you have 50 members all with the current highest rank before creating a new one. It is likely to be quite some time before you get to that position!

Hopefully some of the other members will add their own valuable ideas to this discussion.

I like the suggestions for new certificates.

I have signed up to the e-mail subscription but don’t understand why you would want to take discussions away from the main forum, making it less likely that other people would see them and hence preventing them from making a contribution.

In view of my previous comments I feel inclined to suggest a new problem and would like to start with a suggestion for a fairly easy one. However, I will wait until you are ready to accept new suggestions first. I am also somewhat confused as to where you would want those suggestions to be made.