Ah, so
that's what the problem is. Where in the game are you, story-wise? Because you seem to have skipped a decent sized chunk.
Go back to Fynn. Talk to Hilda, and learn the tearm
Ekmet Teloess. talk to Paul to find out where the door is. The White Mask is under Castle Fynn. Then, once you've gotten said mask, go to Mysidia and put the White Mask on the statue. Then go back to the Cave of Mysidia and put the Black Mask on the doppleganger. You'll get the Crystal Rod out of that dungeon, which you need to get into the Mysidian tower.
Hope this helps.
Spoiler: And I'd recommend de-equiping Lela before entering the atoll that the tower is in.