Day after day, I would see him: an old man in shabby clothes who used to stand at a busy junction near our house in north London. He held in his hand a piece of cutlery (usually a fork, I think), which he waved high in the air as the cars and buses roared by, sometimes with enormous energy and sometimes quite calmly.

He didn’t seem unhappy but he was certainly a solitary figure, unaware of the people who passed him by. He was in his own world. I assumed – rightly – that he had dementia. Then he disappeared.

The obituaries that followed revealed that he had been a psychoanalyst and brilliant musicologist, whose research had added significantly to our knowledge of how Beethoven and Mozart composed their music. Only then did it occur to me that perhaps with his fork, his spoon, he had been conducting the traffic to the music in his head.

This image has always haunted me because of the great disconnect between what we all saw – a man with dementia weaving around on the pavement – and what he was experiencing, which I like to imagine was both the music of the composers he had so loved, and also a sense of centrality and of control over his world.

What goes on inside the mind of another person is always a mystery, but with very advanced dementia, mystery becomes terrifyingly impenetrable; twilight to pitch dark.

One of the gifts of art is to enable us to enter into other people’s lives and selves. The art that attempts not simply to observe but to inhabit that desolate place of self-loss becomes like an emotional modernism, in which there is no central narrator, no coherent story, where things are fractured and the safe ground slides away beneath our feet.

Exploring the experience of dementia and the loss of memory can bring about a powerful, and vertiginously unsettling, way of thinking about time, place and identity, where the notion of a stable reality and a single self breaks apart. Frank Kermode called it “decreation”, where words and meanings are unmade.

There are few people now who have not had some experience, however distant, of dementia. It’s all around us, among us, perhaps inside of us. When I was little, I saw my grandmother, at what I now realise was an early age, gradually move into a sweet-natured fog, a benevolent dissipation, but because I was young it was a mild experience – like seeing a pencil drawing slowly rubbed out until only a few vague lines were left.

A few years later, my grandfather on the other side of the family was also affected by the disease, and this time I understood more of its power. I also felt it as a kind of violation of the man I loved and looked up to.

My father also had dementia. I know it is in my family, perhaps in my genes. So when I point to a thermos flask and repeatedly call it a Tupperware, when I pour ground coffee into the dishwasher-tablet capsule or am unable to remember the name of someone I know very well, when I feel I’m in a bit of a muddle as if I’ve become disconnected from some power source in myself – then I fear that this is a harbinger. This is a special way of being afraid: not just Philip Larkin’s furnace-fear of death, but fear of a profound loss of self.

Over the past seven months, I have been campaigning for the rights of care partners to accompany patients with dementia in hospitals, I have visited hospitals up and down the country. I have seen how some people can live well with dementia – an immersion in the present moment which some people even experience as a gift.

Now is all there is. It’s crucial to use language that doesn’t cast those with dementia into the role of victim: they are not “sufferers”; those who care for them are not simply their carers but remain their spouses, their children, their friends. (Yet, I had to do a bit of editing to remove this type of language from this article…)

Dementia is not a monolithic condition, a simple sentence of doom, and people with it are not “bed-blockers”; they are human and they are precious. Yet, at its most advanced, it can feel uniquely distressing. I think of the woman in one ward who called out over and over again, “No, don’t, please don’t” – stuck in a loop of distress that no one knew how to break. I imagine she was perhaps reliving some kind of trauma: it looked like a kind of hell to me.

There was another woman, tipped sideways in her bed, thrashing, shouting for help, though what she wanted was impossible to tell. On the cabinet to her side was a photo of herself and her husband standing by the sea when they were young. She looked beautiful and glad, her whole life ahead of her. What would that carefree young woman have thought if she had looked ahead to see the person she would become?

There is so much loss around dementia, so much sorrow; an existential scariness and an infinite strangeness. Who are we when we no longer have our memories? At what moment do we die? In the New Yorker, writer Stefan Merrill Block quotes David Shenk’s The Forgetting to describe dementia as a “radical slowing of death” – “what is usually a quick flicker, we see in super slow motion.”

Samuel Johnson once said that art enables people “better to enjoy life or better to endure it”. Illness has always been a subject for art: think of Tolstoy’s Ivan Ilyich, Chekhov’s short stories. Illness throws us out of our normal lives. Different illnesses become relevant at different times, like Holden Caulfield’s nervous breakdown in the 50s or AIDS in the 1980s, in works such as Thom Gunn’s The Man With Night Sweats. Over the last few years, as the population ages and the number of those who have the illness dramatically increases, dementia has entered our culture and our consciousness.

From being hidden and largely invisible it is finally being talked and written about, it is acknowledged. John Bayley wrote an elegy about Iris Murdoch’s loss of memory; Sally Magnussen wrote a groundbreaking book about her beloved mother’s dementia; Timothy West has been moving about the Alzheimer’s of his wife, the actor Prunella Scales. Terry Pratchett was heroic in his openness about his own dementia.

Sometimes I have wondered if Haydn’s “Farewell” symphony – in which one by one the musicians lay down their instruments and leave the stage, music gradually thinning into silence – is an evocation of it. There is, of course, the melancholy Jacques’s seven-ages-of-man speech in As You Like It (starting with the “mewling and puking baby” in its nurse’s arms and ending with a “second childishness” of decrepitude: “sans teeth, sans eyes, sans taste, sans everything”).

Or King Lear, who endures a storm that is both literal and psychological, whose identity crumbles and words fall away, and who dies deluded (when Ian McKellen played him a few years ago, he dropped his trousers and stood on the stage befuddled, stripped down to a “bare, forked animal”). Friends have pointed to the Aged P in Great Expectations; a character in Trollope’s first novel (“senile at 50”); maybe Mr Wodehouse in Emma; perhaps Don Quixote.

Recently, however, depictions of dementia have become so common in art that they are almost a trope. Films such as the poignant Away From Her, starring Julie Christie, or the Oscar-winning Still Alice, have characters with Alzheimer’s at their centre. Ian McKellen, again, plays the great detective in Mr Holmes, towards the end of his life when he can no longer remember his cases: we know his stories when he has lost them. The final series of 24 showed, in a rather cheap fashion, a US president who was in the early stages of the disease.

Henning Mankell’s last Wallander novel, The Troubled Man, has the detective suffering from memory loss that makes “whole chunks of time just disappear. Like ice melting away.” The investigation into a crime is accompanied by a parallel investigation into the increasingly shadowy interior. But these are all realist texts that remain stable in spite of their subject matter. They can’t resist the notion of plot, order, a single coherent self.

Dementia might be the subject but it does not disrupt the narrative. Emma Healey’s impressive debut, Elizabeth Is Missing, takes us closer to the mind’s disintegration. Its narrator, Maud, is 90 and has memory loss. Yet, she knows that the remains of a compact mirror found in a friend’s garden is a clue to a 70-year-old mystery that has always haunted her. And she knows, too, that her dear friend Elizabeth is missing, although no one will tell her why. Her mind holds on to these two things, even as she forgets almost everything else. Language slips, words can’t be fitted to things. Things loom at her out of a fog of unknowing. Faces have no names. What she sees is held within a tiny frame of seconds, with no before and after, so that she – and we – see the world in an agonised and incomprehensible close-up.

Yet, in the end Maud solves both mysteries, and this radical memory loss is really a well-managed device. Maud is the unreliable narrator par excellence. She joins Mark Haddon’s boy with Asperger’s syndrome (Christopher John Francis Boone in The Curious Incident of the Dog in the Night-Time), whose literal interpretations of human events take him on a funny, heart-wrenching journey, or SJ Watson’s Christine in Before I Go to Sleep, whose memory is nightly erased by sleep and who faces each new day with an appalling freshness, being stranded in the present.

These unreliable narrators – the wonderful Saga in the Danish series The Bridge is another – have the knack of finding out the truth that more rational characters cannot discover. Even in Samantha Harvey’s fine novel The Wilderness the character with dementia is looking back through a thickening fog of forgetfulness at his more vivid past in order to confront loss; he can hold on to a vision that endures. It’s an alluring thought that damage can offer enlightenment and that profound loss has a consolation. Like Benjy in Faulkner’s The Sound and the Fury, “mad” people can see the world in ways that the “sane” are blind to; they can illuminate the human condition.

In the TV series The Fear, Peter Mullan plays a Brighton crime boss struggling with memory loss. His profound uncertainty infects the dramatisation, which is full of strangeness and menace. We experience the world through Richie Beckett’s eyes: we see characters morph into other characters; we hear words sliding away into a fuzz of sound. Cruelty erupts out of tenderness. Shapes waver and boundaries rupture. The world becomes indecipherable, a modernist drama made up of fragments that we have to join together.

Matthew Thomas’s extraordinary We Are Not Ourselves (whose title is taken from King Lear) comes intimately close to the horror of the mind’s disintegration: it is an epic account of the history of a family, beginning with Eileen Tumulty as a child in an Irish-American home in Queens, New York, and following her through her marriage to Edward Leary, the birth of their son, and her husband’s slow decline. The triumph of this sprawling, magnificent novel is its portrait of a disease that makes its way insidiously into the pages, so that for many chapters neither the characters nor the reader understands what is happening. Bit by bit, like mist obscuring a familiar landscape, communication becomes obscure and incomplete.

Careful and courteous Ed Leary calls his wife a “bitch” and becomes an angry, despairing man, stumbling about in a world that has lost its signposts and contours. Even this nightmarish flux and shapelessness can only endure as long as Ed Leary holds on to a knowledge of himself. Gradually, he falls away from his wife and the reader, moving from the edge of darkness into its centre. Incontinent and inarticulate, he becomes a mute object in the story rather than its disintegrating subject. He cannot speak; there is no “I”. The narrative reverts to Eileen, who watches and guards him with her fierce and generous love.

In Florian Zeller’s play The Father (translated by Christopher Hampton and seen recently in a terrific production at the Tricycle theatre, and transferring to the West End in the autumn), the central character, played by Kenneth Cranham, has Alzheimer’s. It starts calmly, with a comfortingly ordinary, bourgeois living room: bookshelves, table, armchairs, pictures, a man talking to his daughter. We realise that he is losing his memory, but, at first, we are simply observing an all-too-familiar family drama.

Within minutes we are wrong-footed; the ground gives way. Whose flat are we in – his or hers? Is she his daughter, and, if so, who is this other woman who says she too is Anne? Who is the man who says he is Andre’s son-in-law, and why are there two of them? Was Andre a dancer or an engineer in his professional life? Why does the dinner of roast chicken keep repeating? Where is his watch? We never know the answers because confusion is the point (“It doesn’t make sense”, says Andre over and again). Between short scenes, interspersed with increasingly jarred and stuck music, the furniture on the stage gradually disappears. In the end there is just a white-sheeted bed in a white-painted room, a man calling for his mother. The audience both inhabits Andre’s mind and at the same time tries to decode its erratic signals, for The Father – beautifully, heartbreakingly and exhilaratingly – continually reminds us of our need to make a story out of chaos while brutally undercutting it.

The London-based American artist William Utermohlen was diagnosed with Alzheimer’s in 1995 at age 61. H continues painting his self-portraits over the next five years. In the first, he is recognisably himself, although his thin face has an anxious, watchful expression. But quickly the perspectives flatten, and the spatial sense is lost. The face, in the final portrait, is simply a scribbled head.

Erwin Mortier is a Flemish writer, a poet and novelist. He watched his mother living with Alzheimer’s, and, out of this anguish, came his Stammered Songbook, which is both an unsparing observation of her disintegration and an evocation of it. It is about the decaying body (ribcage and cavities, folds and membranes, lung tissue and the convolutions of the brain) and also about the decaying mind. At the heart of this powerful meditation is Mortier’s struggle with language and his attempt to communicate that which lies beyond communication. Many of the pages contain only a few broken lines, surrounded by white space, by absence and silence that words cannot fill.

Mortier repeatedly reaches for metaphors: a house that is burning, a house that is collapsing, a capsizing ship, verbal rubble and grammatical ruin, a whirlpool, a coward that giggles behind the cupboard and under the floor, an ice floe breaking up, a stagnant fen, a tide going out… Each metaphor is devastating; none can capture the “de-wording, de-languaging, de-remembering”. We can only stand at the threshold of the silence, the darkness and the absence.

George Oppen was an American poet, long overlooked and now revered. An exponent of objectivism, he sought to communicate the thingness of things; his poems travel between word and object and are precise, unsentimental and intelligently beautiful. His last collection, Primitive (1978), was written as Alzheimer’s began to take hold of him. Amyloid plaques and tangled brain neurons. In one of these final poems, Disasters, he writes of “the caves/of the hidden/people”. Soon, he too would be one of these hidden people, in that cave where light doesn’t reach, and from which no signals return to us, who stand at its mouth gazing in. The rest is silence.

Yet still the old man waves his fork into the air. Perhaps he hears music.