Further explaining their relationship, he adds: "There has been the odd barney over the years, but it's always me - I cause them. It's me being hotheaded, because I just am. It doesn't matter if I'm working for Paul McCartney, the Pope or whoever.
"I'm a bit intolerant of authority. But if Paul McCartney was a dickhead I wouldn't work for him."
He pauses to consider his role in life: "I think it's hysterical that I work for someone who is the most famous person in the world apart, perhaps, from the Pope - although the Pope's songs aren't as good.
"But it'll probably end very soon - probably after Paul reads this! I really wouldn't be surprised if it ended this year."
You've been doing the job for 14 years, so why should it end now? "Because I don't presume anything. I keep having to change the remark, but when I'd worked for Paul for 10 years I said: 'If you work for Paul McCartney for 10 days that would make your life.' I'm not saying that in a sycophantic way, because he gets enough of that.
"But it's ******* fabulous. It's a privilege; it's not work. It's been a blast. And that's what I think it's meant to be; fun and a laugh - it's only rock 'n' roll."
We are, at this point, sitting on the steps of the Abbey Road studios. Some tourists are gawping at us. One is probably saying: "Look at that scruffy soand-so - it must be Geoff Baker."
Non-stop Geoff - possibly powered by Duracell batteries - does wonderful impressions of a hyperactive child and a runaway train at this world-famous musical landmark in London's St John's Wood.
He has, for once, agreed to step out of Paul McCartney's shadow and into the spotlight - but only because it will give him the chance to further promote the new, stripped-down version of The Beatles' album, Let It Be (Let It Be ... Naked is out today).
Geoff, who is great company and one of the most enthusiastic people I've ever met, is excited. Very excited. Over the course of more than four hours, he will tell me and several others "The Beatles are on the ******** front cover of the NME, for the first time since they split up! How cool is that?"
Crews from BBC News, Radio 4, Associated Press TV News, Sky News and a French TV station are here to interview Paul Hicks and Guy Massey, two of the producers who remixed the album using Abbey Road's digital technology (Allan Rouse was the third).
Sir Paul, naturally, is at home with his wife, Heather, holding the baby.