Sir Alex Ferguson should stop complaining about referees: David Cameron should shut up about judges. Attacking them rarely pays dividends in the long run. Look at the fate of David Blunkett.

The populist politician who took on Lord Woolf is pretty much forgotten. His onetime adversary is still going strong – currently returning to public life to check the validity of Saif Gaddafi’s doctorate.

The trouble with the Conservative element of the coalition is that they are fixated by what they see as the Blair playbook. Apparently, they can quote the former prime minister’s ghastly Journey at length.

In August 2005, the master famously announced, in the context of counter-terrorism legislation, that ‘the rules of the game have changed… Should legal obstacles arise we will legislate further, including if necessary amending the Human Rights Act’. It has been taken as a telling sign of strength.

Thus, Cameron pounced on a six-year-old decision of the European Court of Human Rights about the right of prisoners to vote, announced that it made him ‘physically ill’, and followed his hero into a charge against the Human Rights Act.

A subsequent decision of our own Supreme Court – that sex offenders should not automatically be kept on the register for life – was treated as petrol on the fire. We now have a revival of the Tory manifesto proposal to review the Human Rights Act.

Labour’s response to this threat to its flagship legislation was incoherent.

Jack Straw joined forces with David Davis to assert parliamentary sovereignty against the European Court of Human Rights. It was left to John Prescott to defend the need to follow the court’s judgments – albeit so incoherently and incorrectly that I cannot have been the only listener who felt that they had to switch off the Today programme when he did so.

Nevertheless, and not for the first time, Lord Prescott was essentially right. The UK signed up to the European Convention on Human Rights; this binds us to implement final decisions of the European Court of Human Rights; we support the substance of human rights and wish to uphold the convention.

Policy Exchange, the right-leaning thinktank, fanned the flames by the timely publication of a paper by political scientist Michael Pinto-Duschinsky. This concludes that ‘the UK should consider withdrawing from the jurisdiction of the European Court of Human Rights in Strasbourg and establishing the Supreme Court as the final appellate court for human rights law’.

Pinto-Duschinsky prays in aid the example of Australia and New Zealand, which have renounced the authority of the Privy Council in recent years: ‘Entirely understandably, they have decided that they would like final decisions on key questions to be made by judges in their own country.’ Neither country is, however, a member of the Council of Europe.

There are some calming voices to be heard.

Lady Thatcher’s long-time lord chancellor, Lord Mackay of Clashfern, counselled a sense of perspective in giving evidence to a parliamentary committee: ‘When we have taken on legal obligations, as we have under the Convention on Human Rights and Fundamental Freedoms in Europe, the rule of law requires us to obey the courts that are set up under that convention whose judgments are binding. I do not myself regard politics and law as, in any way, in conflict because politics in our country should be carried on under the rule of law.’

Ken Clarke, arguably the best leader the Tory party never had, was characteristically relaxed about the consequences of the European Court judgment when pressed by Jeremy Paxman on Newsnight.

More calm is desirable. Article 46(1) of the European Convention on Human Rights binds high contracting parties ‘to abide by the final judgments of the court in any case to which they are parties’. Article 46(2) states that ‘the final judgment of the court shall be transmitted to the committee of ministers, which shall supervise its execution’.

The ultimate sanction would be to expel or ‘invite withdrawal’ of a recalcitrant enforcer of court decisions under articles 3 and 8 of the Statute of the Council of Europe for breach of the principles of the rule of law, serious breach of human rights or failure ‘to collaborate sincerely and effectively with the aims of the council’.

You can argue about the consequences of membership of the EU but we would probably be expelled from that also for breach of fundamental rights.

Do we really want to give the issue of prisoners’ rights to vote and sex offender registration such prominence? The justification for removal of the vote is the feudal idea of civil or civic death, revived by the Forfeiture Act 1870.

An offender becomes, in the medieval description, an outlaw who sunders ‘every bond between society and the man who has incurred it; he has ceased to be a citizen’. In 2009, all but 34 prisoners on ‘whole-life tariffs’ knew that they would return to the outside world. Because of this, we encouraged continuing contact with families for the other 86,000.

It makes equally good sense to encourage prisoners to exercise their duty – not right – to vote. Scotland quietly let sex offenders have a minimal right of appeal; no one really noticed.

Manchester United may yet win the Premier League and David Cameron an outright majority at the next election. Both might remember that referees and judges merely hold the ring.

There is plenty of scope for an adequate range of political response within it. Stop the showboating and quietly work out an adequate response to judgments that require you to recognise that the concept of those outside the reach of law or politics needs some updating.

Roger Smith is director of the law reform and human rights organisation Justice