The article examines the power of administrative bodies to assess the constitutionality of legislation (‘administrative constitutional review’), taking examples from Swedish public law. The Swedish constitution explicitly requires all public bodies to engage in administrative constitutional review when necessary. In this way, Swedish administrative authorities have the right and duty to act as guardians of the rule of law. This competence relates to the historical development of Swedish public law, which deviates from most other European constitutional systems by organizing all state administrative authorities as separate public organs detached from the Government and the ministries. The Swedish constitutional obligation is parallel to EU law requirements on national administrative organs to set aside national legislation in conflict with directly applicable EU law (‘the Costanzo obligation’). Against the background of practical examples in Swedish law, the article identifies theoretical and practical challenges for administrative bodies to engage in constitutional review. These include the risk of disturbing constitutional structures by putting lower administrative authorities on par with the parliament. The possible problems of lack of legal expertise and the problem of independence in practice are also discussed. At the same time, the concept of administrative constitutional review has a potential to protect the constitutional system, including the fundamental rights of individuals.