In this paper, we propose a semi distributed approach, for load balancing in large parallel and distributedsystems. Theproposedschemeisa twolevel hierarchical scheme which partitions the interconnection structure of a multiprocessorsystem into independentsymmetricregions. Weconsiderinterconnection structures belonging to the classical infinite familiesof distance transitive graphs. The proposed schemeusesthepartitioningpropertyof thesegraphs by employingHadamardmatrices. Theperformance of systems consisting of these interconnectionstructures with varying sizesis evaluated through simulation and compared with a fully distributed scheme. Simulation results indicate that the performanceof the proposed scheme improveswith the increase in system size.