Hey! There's no reason why a city of learning and wisdom shouldn't have several universities. Look at Oxford in the UK. Its not really more than a big town, but it has no less than 38 'colleges' (known anywhere else as universities )

The same is sometimes true of libraries - though not in quite such profusion. Before the internet there were several small specialist libraries in my own home town, which only has a population of 53,000.

Don't be put off including whatever you want to include. Its your plot, and you have just as much right to do a thing as anyone else