The control of power networks through physical means, as well as through regulation and price mechanisms that incentivize distributed decision-making, present tremendous challenges that may significantly impact the economy, the well-being of nations, and the world as a whole. Grids are loosely-coupled, dynamically interconnected domains with communication, computation, and control functions at multiple temporal and spatial scales. Anticipated societal benefits include mitigation of climate change and improved energy security. Consumers, markets, and regulators acting as participants and stakeholders exacerbate the challenge. It is expected that in coming years the mathematics of stochastic control will be more central in providing insights into system-level, architectural, and policy issues.