IIM-A was established in 1961 as an autonomous Institution by the Government of India in collaboration with the Government of Gujarat and the Indian Industry. Dr. Vikram Sarabhai, a noted scientist and industrialist, other Ahmedabad-based industrialists (led by Mr. Kasturbhai Lalbhai), and the Government of Gujarat (led by Dr. Jivaraj Mehta) played a major role in the creation of the Institute. IIM-A was registered as a society with a Board of Governors to oversee the functioning of the Institute. The Board has representatives from the Governments of India and Gujarat, the Industry, the IIM-A Society, the IIM-A faculty, etc.

IIM-A was conceived not to be purely a business school, but a school of management. It's mission is to professionalise Indian management through teaching, research, training, institution building and consulting. It also aims to professionalise some of the vital sectors of India's economy such as agriculture, education, health, transportation, population control, energy, and public administration.

In its formative years, IIM-A collaborated with the Harvard Business School (HBS). Largely as a consequence of this collaboration, IIM-A pioneered the case method of teaching in India.

Not only did IIM-A adopt a distinct pedagogy, its architecture also took a distinctive form. Louis Kahn, a well-known American architect, conceived the design: a blend of austerity and majesty; spaces for casual interaction; frequently changing perspectives; a balance between modernity and tradition that captures the spirit of contemporary India. His vision took the shape of a vast ochre temple of learning on a 60-acre site that attracts visitors from all over the world.