Once you have set up the 8 Settlements necessary from Form Ranks this quest can be initiated by either speaking with Preston Garvey, listening to Radio Freedom or visiting The Castle.
Listening to Radio Freedom will give you informaiton on the direction from where the attackers are coming from. When defending the Castle, pay attention to the state of the generators, and Synths who will teleport through walls. It is a long and arduous battle, build your defenses carefully in preparation.
Once the Institute has been repelled speak with Preston and Ronnie Shaw.