There is some humour in Proust. He does not write comedy but there are some comic descriptions. They are not laugh-out-loud moments but they do make you smile. There are no particular set-ups to let you know a gag is coming and the comic moments are more surprising and pleasing for this. It does mean that you have to pay attention in case you miss anything.
I wonder whether this is a comic novel in the way that people describe Joyce or Beckett as comic; where comic does not mean funny but rather describes an authorial view of existence as a farcical struggle against combined elements of destruction that oppose characters' every scheme and intention. This gets reflected in modern comedy writing where it seems that comedy can only be found in exposing a character's vanities or delusions and having their every effort exposed as failure. Perhaps it is not possible to create comedy out of success, or just competence, or perhaps it is just easier to make failure funny.
Proust also makes his comedy out of exposing his characters' failings, like Jane Austen does. For example, the Cure is telling the narrator's aunt about a couple of tapestries in the church and he says that "I can quite see too, that apart from certain details which are - well, a trifle realistic, they show features which testify to a genuine power of observation". This conveys not only the overly prudish sensitivities of the cleric but also his lack of awareness of the meaning of what he says in complaining about the tapestries' excessive realism while praising the artist's observational skills.
There is another incident earlier in the novel where the narrator describes how his aunts (I think) are so keen to be subtle in thanking Swann for a minor kindness he has afforded them that, rather than simply saying "thank you", they are overly literal in the nods and winks they offer him so that it is obvious to everyone in the room what they are doing apart from Swann.
Not thigh-slappingly hilarious then, but there is humour there.