Still feel like a dirty little secret years after affair started

I've been with my partner for 10 years and we got together by having an affair - he’s 10 years older than me and has four children with his ex, ranging in age from 14 to 21.
When his wife found out about our affair, she kicked him out and he went to live in a flat but we now live together.
The issue is, he still hasn't divorced his ex and he also sees her way too much in my opinion, often just popping round to hers, and he's constantly on the phone to her about the kids or stuff to do with their family home.
He doesn't involve me in any of it. I've met his children only a couple of times and they've never come to our place. I'm not a part of any of it.
While I realise we have to tread carefully with his kids, we've been serious for 10 years now and I'm still made to feel like his mistress or his dirty little secret.
Whenever I try to talk to him about how I feel, he guilt-trips me about the kids and it’s very much a case of “this is how it's got to be”.
There's no room for any compromise and I'm sick of it.
I'd love your opinion on whether you think he'll ever change and include me properly in his life.
Coleen says I DON'T think he wants to change or compromise, at least for now. It sounds like he's struggling with a lot of guilt.
His wife has had years of caring for four kids largely on her own. He's trying to be there for them but he's failing to make you feel secure.
I imagine you've talked about why they haven't divorced - is he waiting until his youngest is olde...