It is widely recognised, and mostly accepted, that ‘utilities’ provide a public service and not just private services, so it is legitimate to regulate them in ways that ensure the public interest. This is partly because there are always elements of natural monopoly in the way in which these services – power, water, fixed line telephony and cable, public transport, etc – can be delivered. It makes economic sense to only have one set of lines, pipes and cables going into one property, neighbourhood, etc. The other public interest dimension of these services is that they are regarded as essential – rights almost – for survival in a modern society. hence there is usually an obligation for ‘universal service’ which includes provision of service to uneconomic areas (e.g. remote rural locations) and strict conditions on suspension of supply, usually only in extremis.
So far so good, but the question immediately arises how are they to be regulated to meet the public interest, and how far does the public interest extend?
The ‘UK model’ of utility regulation lies at the extreme end of a spectrum. Developed under Thatcher and never ally altered under Labour, this model focuses strongly on competition as the primary outcome and pays only minimal interest to protecting public interest concerns.
It also has specific process characteristics: largely opaque processes, very limited transparency from providers and no requirement for regulators to publish specific reasoning to justify decisions, which often appear to bear no relationship to any evidence.
There is a largely ignored democratic alternative from probably the least likely place one would expect to find it: that bastion of ‘free markets’ the USA.
In their rather neglected book ‘Democracy and Regulation – How the Public can Govern Essential Services’ (Pluto Press, 2003) Greg Palast and colleagues analyse in detail how this system, used in various forms across the USA, works.
Very simply it starts from the assumption that the ‘public’ interest in public utilities needs to be strongly reflected in how they are regulated.
This translates into two simple principles – first that effective regulation requires complete transparency by providers and regulators. No hiding behind ‘commercial confidentiality’ – anyone providing public services must accept full public scrutiny.
In practice this means strong ‘discovery and disclosure’ rules that force providers to divulge any and all information about their operations. At the same time, regulators must also disclose full reasons for their eventual decisions. And there are even requirements for parties who make submissions to regulators, or seek to discover information, to themselves be open about themselves. In other words complete transparency.
The second key principle is strong public (civic) involvement with the regulatory process. This usually means a very open regulatory process where interested parties get the chance to seek information and put views to the regulators, which the latter are legally required to consider.
Public engagement covers not just pricing, but also levels and types of service and standards to be met such as coverage, safety, environmental responsibility, etc.
All of this operates within an assumption that utilities should be treated a bit like government bonds – they should be seen as secure, but not exciting or risky, long-term investment opportunities that guarantee a reasonable, but modest and secure return on investment. This often involves regulators fixing maximum returns for investors.
The net result, according to Palast & co, is that the USA has amongst the cheapest public utilities in the world.
This example could provide some useful yardsticks for the current debate about other public services. For example, if private and non-profit providers are to be encouraged into health then they too should be subject to ‘democratic regulation’ – complete transparency and strong public involvement in deciding prices and quality of services.
STATECRAFT - improving government 
I think that open source software development offers good and challenging evidence of your point. Most open source software follow the GPL licence allowing everyone to be free to have access to the software code, to use it, and to change it on the condition that you give others the same access to your changes. As a rule, open source software is developed under high transparency, and indeed it has resulted in many very fine software packets (firefox, apache webserver, openoffice, r-project statistics packet, etc – all of these are extremely complex). By redefining copyright, open source software creates a public good outside the state, a creative commons as Lawrence Lessig has termed it (www.creativecommons.org). The internet is build on similar principles, as well.
My point is not that all public goods may produced in that way. Open source software is based on the fact that software development is expensive, while the production of copies costs little or nothing. In general, public service is definitely not produced without costs, hence we cannot expect public service provision free of costs. No, my point is that the development of complex service providing systems may indeed follow the principles of openness and public involvement, as you refer Palast for, and that these two principles have a huge development potential (sometimes people are even not paid for their contribution to the development). Of course, there are benchmarking within open source development, and developers are aware of the resources spent, etc. . However, the main drivers of open source software development are not performance measures, but openness and public involvement.
Moreover, the biggest revolution of our time, the internet, seems to support your point. Perhaps you have a good point 😉
I wonder though if there is a constraint imposed by the EU procurement rules, and whether you have considered this and whether there is a way round any obstacles? I’m aware that the rules essentially apply to the procurement stage rather than subsequent contract management, but your proposals would obviously imply a requirement being included in the procurement process that successful providers would have to accept the broader regulatory regime you are suggesting, and the requirement would need to fit within the EU rules.
This may not be an insuperable obstacle, but clearly it’s one that doesn’t apply in the US. There is, as you say, an argument for much broader regulation and indeed in relation to private sector provision of local government services it is arguably essential. There is of course a cost, though. Whereas a public sector service may be regulated through the democratic process within the authority, plus (for the time being) bodies like the Audit Commission, tougher regulation of multiple private suppliers would need a good regulatory team within the LA (or somewhere).
I mention these points not as an objection to the idea – if there is to be more privatisation we should argue for the corollary of proper regulation. In some senses, in my field (housing), the government has accepted the need for a greater democratic/customer role in regulation, but the problem is that so far it looks pretty toothless and tokenistic. This is a great danger that needs to be addressed. Cross-referring to experience elsewhere, such as the US, clearly helps in doing that.