I think a lot of it would depend upon the execution (i.e., the writing itself). The characters and their motivations would be pivotal. It doesn't have to be anything world-shattering, just what makes sense for X character in Y context/ circumstances. The stakes just have to be high enough/ personal enough for the victim to be threatened or tricked into complying. You can always throw in a nice dose of psychological mind-fuckery- gaslighting, etc to sweeten the pot. Maybe the consequences of whatever getting out would not be that bad to a rational person, but if the victim is vulnerable somehow, or their rationality/ confidence/ beliefs etc can be rocked- maybe the blackmailer makes the consequences seem much worse than they actually are, etc. It's all in the writing and suspension of disbelief. Readers want to read this- they want to suspend their disbelief and get lost in the world of the story, root for the good guys, boo the bad guys, etc. It's up to the writer to keep that illusion up and not jar the reader out of it.
There's also different types of blackmail beyond the "I've got X pictures and I'll release to everyone if you don't to Y." It could be desperation- victim needs something, blackmailer has it (whether that thing is emotional, financial, material, immaterial.) Like, "wanna keep this job? You'll wear a diaper." "You're ugly/ fat/ stupid/ etc. No one would love you. Without me, you'd be alone. Now, if you want me to stay in a relationship with you- and I really put up with a lot of your crap, you know. you could at least do this one little thing for me. I don't ask for much; just put this diaper on..." "I've got the market on meth cornered in this one horse town. I'm the only dealer. You wana hit? but you have no money. How bad you craving it, baby? I'll give you some...I'm a nice person. You just gotta do one itty bitty thing for me...." "Don't want me to show super rich wifey poo this video of you cheating on her? Y'know, you signed a prenup. Divorce and you get bupkiss, bud. It's back to skid row for you. Unless, of course, you listen to me...." "Gee, such a shame if all this evidence landed on the District Attorney's desk. You'd be arrested so fast your head will spin. Oh, you're acutally innocent? Wanna take a chance and prove it in court? I could make all these nasty charges go away...I just need something from you, too..."'
It's a power exchange- blackmailer having power over the victim, and for all kinds of different reasons and scenarios.