I understand the phrase slightly differently. "Toffee" here means something of small value. A variation would be "tuppence" (i.e two pence), as in "he can't ride for tuppence". So the basic meaning is, he has so little riding skill it isn't even worth tuppence, or in this case, toffee.
Thank you, this is very useful and I shall be showing it to my wife who eats carrots by the ton. One more thing: is there any truth in the urban myth that if you eat too many carrots you will actually turn orange?