A combination of horror movie and psychological thriller that’s deliciously old-fashioned in technique while putting a new spin on familiar tropes, Jennifer Kent’s “The Babadook” is a rarity, both genre exercise and art film. One could compare it to one of Tim Burton’s animated pictures minus the whimsy, or to “The Shining” minus Jack Torrance (with the mother going the possessed route instead). You can even say that it recalls Roman Polanski’s early work—“Repulsion” in particular. But the fact of the matter is that it has a distinctive voice all its own.

It’s essentially a small two-character piece in which mom Amelia (Essie Davis) and her six-year old son Samuel (Noah Wiseman) live still traumatized by the death of Amelia’s husband Oskar (Ben Winspear), who was killed in a car crash while driving her to the hospital to give birth to the boy. Samuel’s a high-strung kid, obsessed with protecting his mother and constructing a barrage of weapons for the purpose. In fact, when he takes one of them to school, it’s the final straw in a series of episodes involving his classmates that induces the authorities to determine that his behavior will need to be constantly monitored, a decision that leads Amelia to withdraw him from the place. That will eventually lead to the intervention of social services.

Though there are a few other characters involved as the plot unfolds—Amelia’s unsympathetic sister Claire (Hayley McElhinney) and her bratty daughter; elderly, helpful neighbor Grace (Barbara West); Robbie (Daniel Henshall), Amelia’s pleasant colleague at the nursing home where she works. But Kent’s script introduces them mainly to upend your expectations about how they’ll figure into things. When push comes to shove, there’s only one other figure, besides Amelia and Samuel, who matters—Mr. Babadook, the title character of a children’s pop-up book that mysteriously appears in the boy’s bedroom one night.

Who is Mr. Babadook? That’s the question. He can be described as a ghoulish fellow who owes a lot to German expressionism and Edward Gorey, though perhaps the closest physical approximation would be the fake vampire played by Lon Chaney, Sr., in Tod Browning lost 1927 movie “London After Midnight,” complete with stovepipe hat and sinister grin—at least if the surviving stills from that film don’t lie. But what’s important about him is that as he appears in the book, a sumptuous item equipped with thick pages and little levers that make some images jump out at you, he comes across as a threatening sort who, once admitted after knocking three times, will be impossible to get rid of and means you no good. Amelia’s taken aback by the book and tries to destroy it, ripping it to shreds and burning the remnants, but that doesn’t work. It reappears on her doorstep, carefully pasted back together and more menacing than ever. And as if that weren’t enough, Mr. Babadook shows up outside its covers, most memorably one night in shadowy form in Amelia’s bedroom, and occasionally drops a guttural message on her phone. The frightened woman takes her concerns to the local police, but the scene that follows in the station is as scary as anything happening in her house.

Gradually the experience affects the isolated mother and son in fearsome ways. The boy becomes more and more a handful, alternately demanding attention and shrieking. And Amelia begins to lose her patience with him, feeding him tranquilizers while trying futilely to sleep herself. Her attitude deteriorates so radically that she actually seems bent on hurting him, which leaves him no choice but to defend himself against a person he no longer recognizes.

Of course the question is whether Mr. Babadook is some sort of evil entity that’s taking over Amelia, or the manifestation of her inner psychological torment—something like the Krell in “Forbidden Planet.” Whichever option you choose, Kent’s film delivers in the same way as the hallucinatory George Melies clips that keep popping up on the family’s TV along with a plethora of excerpts from old horror flicks. Davis and Wiseman are both astonishingly effective, causing you both concern and sympathy, and Kent, cinematographer Radek Ladczuk, production designer Alex Holmes, art directors Holmes and Karen Hannaford, set designer Ross Perkin and decorator Jennifer Drake, and editor Simon Njoo assiduously build up an almost palpable atmosphere of dread.

“The Babadook” has its share of shocks, but not of the cheap variety: it insinuates more than it goes for the jugular, and ultimately offers the rather grim observation that in the end we might just have to learn to live with our demons rather than exorcizing them. It’s a stylish, sophisticated horror fantasy that provides some genuine scares and, more importantly, a constant stream of shudders, the sort of unsettling genre piece that will stick with you far longer than its bloodier, more gruesome cousins.