Kathleen Coessens 1
A man sets out to draw the world. As the years go by, he peoples a space with images of provinces, kingdoms, mountains, bays, ships, islands, fishes, rooms, instruments, stars, horses, and individuals. A short time before he dies, he discovers that the patient labyrinth of lines traces the lineaments of his own face.
Jorge Luis Borges 1960, p71.
Marco Polo describes a bridge, stone by stone. 'But which is the stone that supports the bridge?' Kublai Khan asks. 'The bridge is not supported by one stone or another,' Marco answers, 'but by the line of the arch that they form.' Kublai Khan remains silent, reflecting. Then he adds: 'Why do you speak to me of the stones? It is only the arch that matters to me.' Polo answers: 'Without stones there is no arch.'
Italo Calvino 1974, p66.
The visual is often considered as a subject of scientific investigation or of aesthetic consideration. In the former case, a physiological and cognitive explanation of vision and the brain offers insight in this complex perceptual apparatus and establishes the relationship between the experience of seeing and the underlying mental processes and perceptual structures. In the other case, the analysis of norms and values of creativity and aesthetic reception determines prevailing or changing cultural settings, exploring the link between the artwork and its position or 'representation' in society. The relation here is that between the image or the 'seen' and the outside — cultural — world. Both inquiries presume some direct relationship between the object and the knowledge behind the object, respectively the 'seen' and the perceptual-mental apparatus, understanding human perception, or the 'seen' and the underlying culture, understanding art. Both rely tacitly on human practices and subjective experiences containing complex meaning-making aspects.
It is the role of semiotics to explore the nature of these meaning-making aspects and to look at the hidden processes of mediation between the object and the knowledge behind it. Semiotics can inquire into the structure of the sign, looking at visual signs as static carriers of visual information. However, it can also consider the dynamic processes of visual semiosis taking place in the visual act between the visual and the viewer, integrating the inherent movement in space-time (Zeller 2007).
In the following, the hypothesis of a visual praxis as an embodied praxis and semiotic process will be developed and sustained by artistic, ecological and cognitive interpretations. In the first part, I will consider Paul Klee's reflections on movement of and in the visual creation and image. In the second part, this artistic view will be met by ecological and cognitive theories of the visual and the notions of affordance and extension. In the third part, position and disposition in the relation between viewer and viewed will be considered. In the last part, Barthes' subjective view on photography and Peirce's tryade of interpretants will offer insights into the human perceptual and embodied position in visual praxis. This position opens a horizon, a continuous process of interpretation and reinterpretation, encountering and triggering an idiosyncratic semiotic disposition of the viewer.
As such, this article explores the meaning-making processes of the human being concerning visual experiences and formulates the hypothesis of the visual praxis as an integral part of the extended body, linking movement in the visual with the movement — in the double sense of embodied movement and emotional movement — of the viewer, position with disposition.
Movement in the visual: seeing as moving
An active line on a walk, moving freely, without goal. A walk for a walk's sake. The
mobility agent, is a point, shifting its position forward.
For the painter Paul Klee, material enters the visual space in a dynamical way, creating a visual praxis. In the first part of his Pedagogical Sketchbook (1972) he looks at the static dot as dynamically evolving into a line that walks, circumscribes, creates planes and surfaces. The line becomes a measure and a basis for the dynamics in the world. The evolution of a line towards some expression, some form is pure movement.
Klee stresses not only the movement which precedes the drawing but also the movement that is inherent as part of the world which the visual image only artificially fixes. Objects in the visual space take three forms of movement: active and moving, passive and being moved or in rest, and finally medial, offering a tool of transmission between what is active and what is passive.
In the second part of his pedagogical sketchbook, the line creates the object, because of the position of the human being in the world and because of the subjective power of the human's eye. Human's power is twice creative and twice constrained, once by its productive actions, once by its receptive actions:
The work as human action (genesis) is productive as well as receptive. It is continuity. Productively it is limited by the manual limitation of the creator (who has only two hands). Receptively it is limited by the limitations of the perceiving eye. (…) The eye must "graze" over the surface, grasping sharply portion after portion, to convey them to the brain which collects and stores the impressions. (Klee 1972, p33).
The third part concentrates on the limitations of this position, on the constraints imposed by the human body and its way of perceiving and moving, and by the world. We are physiologically upright poles with eyes at the top. Being constrained to specific human-based movement and position, humans are constantly adjusting what they see to be able to keep their balance and perceive the world in some kind of acceptable proportion:
The contrast between man's ideological capacity to move at random through material and metaphysical spaces and his physical limitations, is the origin of all human tragedy. It is this contrast between power and prostration that implies the duality of human existence. Half winged-half imprisoned, this is man! Thought is the medially between earth and world. (Klee 1972, p54).
We judge dimensions, proportions and distances in relation to our own position. We create horizons and vanishing points in order that the world does not expand arbitrarily around us. The eye and the brain working together obtain a balance. Klee refers to the arrow as a bearer of movement and thought, as linking body and brain: “The father of the arrow is the thought: how do I expand my reach? Over this river? This lake? That mountain?” (Klee 1972, p54).
Klee 1972, p19.
When we look at a line, we see a line, but it started as a dot; when it closes by forming four angles, we look at it not necessary as a line in movement, but as the line of a quadrangle; when it fills the open space of the quadrangle, we immediately jump to the object seen, the quadrangle, without consciously remarking the preceding movements of the line in which linearity is replaced by planarity. The artist constructs the image in a process of emerging dynamic sign structures, so that it will be readable for the viewer as signs (Zeller 2007, p17).
Viewers apparently jump over the dynamic creational forces of the image, as they want to access as quickly as possible to the information it can offer them. Information gathering is indeed one possible function of meaning-making, it is often an important function, but not necessarily the only one nor the most prevailing. Beneath processes of obtaining information or knowledge concerning the content of the image — which can be indeed just seeing it as a quadrangle —, what is seen, triggers other processes of signification: processes of negotiation, communication, emotion, exploration, movement. What is seen is understood as potentially meaningful: as an expression of the outer world — be it nature or humans —, or as an extension of the own body and mind. This stresses an important aspect of the visual praxis: a visual experience implies complex processes of semiosis.
What is interesting in the sketchbook of Paul Klee is his reflection on the visual as a continuous visual praxis, an embodied, moving, dynamic drawing, without any fixity. The materiality of the world is transformed from objects and their relations towards a visual process he discovers in the pictorial space concerning line, tonality and color. The visual becomes a choreographed, improvised, gestural movement through space and time. With Klee, a starting mark ends in the cosmos: the point and the line from which to move into the world and its place within the universe (Coessens e.a. 2009, p122-125).
Movement of the senses: an ecological view on visual praxis
From the very beginning of our life, and evermore until we die, movement keeps us in touch with our world in the most intimate and profound way. In our experience of movement, there is no radical separation of self from world.
Mark Johnson 2007, p20.
Sensory signification per se is intimately bound up with motoric processes of bodily and environmental interaction in an ongoing process of semiosis that cuts across the sub-systemic distinctions of brain, body and world.
Donald Favareau 2002, p10-11.
Klee offers us an artistic point of view into what I call a dynamic embodied semiosis of the visual. The visual itself is lived as an extension, a scaffold, an exosomatic means of the body and the own position in the world: the visual is not 'detached', the body is 'moved', 'touched' by the visual, the body 'moves', 'touches' the seen by the visual encounter. The visual is an important medium for encountering the world, acting in it and making sense of it.
In recent years, a lot of work has been done and a lot of evidence established on the link between visual perception and action in the world, stressing vision as an exploratory activity. These theories find their origin in the ecological approach of James J. Gibson and in the phenomenological approach of Maurice Merleau-Ponty.
The ecological theory stresses the interaction between the (moving) observer and the environment (Gibson 1966, 1979). Vision is a process of acting in, responding to an environment. Visual perception is an active process in which patterns are detected and skills are acquired by way of exploring the environment. The outer necessary conditions for this are an array of ambient light and an environment containing surfaces and objects with invariant relations in between. The observer encounters regular patterns concerning transformational invariants — e.g. moving away at a constant speed implies decreasing size — and structural invariants — e.g. size constancy, colour patterns, shapes, horizon. He or she can check and double check this by moving, by the reversible transition of occlusion and disocclusion of the objects (Gibson 1979).
The visual is continuously encountered by movement. Merleau-Ponty, in his Phénoménologie de la Perception (1945), explains how we can know what this or that table looks like, because, even if we see it only from one perspective, we have acquired the experience of the other points of view — by moving around. The viewer does not perceive objects in a detached way, but from a bodily point of view: “my body is the pivot of the world: I know that objects have several sides because I can move around them, and in that sense I'm conscious of the world by way of my body.” (Merleau-Ponty 1945, p97). The world is perceived from and inhabited by the body with its motor possibilities and its different points of view (Stjernfelt 2007, p266). The body thus offers different moving points of orientation, limiting as well as opening space-time from its own position.
The invariant or stable information — acquired by kinetic and perceptual experience — guides behavior and is perceived immediately. The body is tuned to an environment containing possibilities, named affordances by Gibson (1979). The affordances are what the environment offers to a perceiver, they concern the possible interactivity between organism and environment. The environment contains patterns and regularities, objects and qualities which are complex, and are integrated as subordinate and superordinate forms with different levels and larger units in the optical array. The visual merges the subject, the viewer and mover, and the object, the seen, together in actions and exploration between body and world.
Today we find interesting developments in the 'sensorimotor contingency theory' of Kevin O'Regan and Alvin Noe. Vision is an 'exploratory activity': “a mode of exploration of the world that is mediated by knowledge of what we call sensorimotor contingencies” (O'Regan & Noë 2001, p940) The exploration of the environment rests on the interaction between the visual apparatus — rules governing the sensory changes — and the character of objects — size, shape, texture, position —, mediated by the active mastery of sensorimotor contingency. The concept 'sensorimotor contingency' refers to the laws which govern the encounter between an observer and an environment: laws are invariant but their mastery complex because of the changing points of view and the different contexts in which these laws are seen under different shapes. This mastery must be currently exercised, leading to latent patterns of behavior and actions of which some are realized in particular situations. Experience and learning processes are necessary to cope with the environment and to adjust what we perceive and how we move, to bring in balance body and world. In some sense we acquire a visual 'habitus': a set of visual principles, a way to cope with what we see in the world, to interact with it, in the first place to be 'tuned' to it or to make sense of it, but secondly, to explore its possibilities. This 'visual habitus' manifests itself as a visual praxis in which the visual opens up to thought and action. As with Bourdieu's social habitus, the habitus which sustains the visual praxis is flexible, adaptable in different circumstances. The habitus is homologous as individuals do not only partake in a culture but have a similar physiological and genetic basis and attune to the same environment. But it is never homogenous because every individual has his or her own life-path, biological and physiological identity and develops his or her own ways of visually and practically dealing with the world (Bourdieu 1980).
By way of this visual habitus — which can be considered as part of a broader perceptual habitus — the visual clues that the perceiver can observe, offer multisensory or multimodal information. The visual perception is at the same time a visual praxis. An example is that of the ideal staircases: Warren showed in an experiment (1984) how there is a close match between optimally efficient riser height and an observer's visual perception of that height: the visual praxis is embodied, offers input for the kinetic. The observer does not need to go to the staircase and try it out, he knows by seeing if it fits. The same observation can be made for reaching the hand in a direction to move a cup: once you have seen where it is, you can take the cup without looking (Merleau-Ponty 1945). These synaesthetic or multimodal ways of approaching the world are developed very early in life by the structures of the visual system, its behavior in an environment and the invariants which are related to these interactions. The newborn experiences the world spatially and temporally by way of facial and gestural imitations and anticipative coordinations (Meltzoff & Moore 1977, Meltzoff & Borton 1979, Bermudez 1995). Three-week-old newborns can visually recognise a pacifier they had previously sucked on — but not seen — from other pacifiers with different surface, revealing the same capacity as adults with the staircase, but inverted — from action to vision. These experiments reveal an inborn capacity of the integration of motor and perceptual capacities, of acting and seeing, which can then be specified by environmental and cultural experience. The first perceptual-kinetic experiences, the first reception of and interaction in the world, offer the newborn clues of time and space, from its own position in time and space (Merleau-Ponty 1945, p162).
The interaction between visualisation and action is demonstrated at another deeper level, namely in the mirror neurons in the brain. The mirror neuron system is a multisensory system that integrates different visual stimuli and converts these into sensorimotor representation (Pineda 2008, p46). The brain integrates different perceptual channels with their respective messages into coherent meaning. Multisensory neurons are involved in many different behavioral, perceptual and emotive processes, leading to a unified percept or experience of an event or object. Input of one perceptual channel is integrated in these multisensory areas and enters a zone of overlap of visual, auditory and somatosensory spaces (Calvert e.a. 2004, p247). By these neuronal integrations, humans fill in the missing pieces and obtain a unified or stable experience of meaning in which patterns emerge. The mirror neuronal system relies primarily on the integration between visual input and motor areas of the brain: it has the ability to remap other's motor states onto the observer's motor representations. Looking at actions in the world, done by others, triggers the same responses and regions in the brain as if the viewer did the action — these neurons are found in the motor areas, but endowed with visual properties. The merging together of the visual and the kinetic is part of our encounter with the world and part of our own biological brain system: they are two sides of the same coin.
A recent experiment with young (six-month-old) children offered evidence for a profound visual praxis: the observation of a moving object activated the sensory-motor cortex of these children, as if they were moving themselves, implying the 'anthropomorphisation' of even nonbiological moving (Shimada & Hiraki 2006, in Gallese 2009, p530). These multisensory neuronal systems integrate bio-kinematic and sensorimotor processes and realise a synesthetic link with corporeal traces or embodied memories of previous social and individual actions and interactions:
our 'grasping' of the meaning of the world doesn't exclusively rely on the cognitive hermeneutic of its 'visual representation' but is strongly influenced by action-related sensory-motor processes, that is, we rely on our own 'embodied personal knowledge'. (Gallese 2009, p526).
These theories offer a cognitive-physiological basis for meaning-making relations between the body and the visual: a process of semiosis takes place on the level of multimodal integrations. Visual perception is always visual praxis, in the first place in a physical, embodied way. As such the body is extended in and by the visual, the movement merges with the 'seen'. Every semiotic interpretation of the visual should take the following three points into consideration.
In the first place, these theories demonstrate that we are by evolutionary means visually 'tuned' to the world, but have to exercise the diverse possibilities and explore the affordances out there so that we acquire a broad range of possible patterns of visual praxis, a kind of 'know-how' or a 'habitus'. We develop a whole range of schemes of a visual praxis: patterns of interaction between seeing and doing linked to contingent and specific situations. This visual praxis is constantly present in the background, being negotiable and flexible, open to new restructuring as foci and situations change.
Secondly, visual praxis means the continuous exchange between an observer and the world as well as between the observer's perception of the world and the observer's proprioception of one's own body. Thus, action has to be seen as taking a move in the world, but also as taking a move with a body, from a specific bodily reference, in which both body and world encounter by way of visual loops — proprioception and perception both inform each other.
Thirdly, visual praxis triggers the body, the mind and perception at a broad level: not only can we observe that there is a continuous reciprocal causation between the perceiver and the world, but even that the world out there is physically incorporated in very complex ways. The theory of mirror neurons proves that seeing others' actions gives you embodied knowledge of these actions — by way of activating these regions in the brain which are responsible for such actions. Seeing thus not only incorporates the world, it incorporates also the actions in the world.
These three points reach further into what I call a dynamic visual materiality or a visual praxis of the world as an extension of the body.
The idea of an extended body and an extended mind goes back to gestalt psychology as well as to Poppers exosomatic means — elements that function as complements or extensions of the human mind and body (Popper 1972). Today its development is mainly addressed by cognitive science, stressing the enormous possibilities of the mind and body to extend themselves by outer means and tools (Clark 1999). Our constraints of memory, physical force, mobility and lifespan, are met by computers, books, weapons, cars, pharmaceutical innovations, and all other products of human knowledge. These products mediate between us and the world, they are at the same time part of the world and part of us and as such blur the frontier between both: they become inherent part of human meaning-making processes.
As such, by way of our visual praxis of the world, are we then extending ourselves, or are we incorporating the world? Both are complementary parts of a whole, of the visual praxis. The recurrent visual loops between I and the world, which direct our continuous bodily involvement with the world, create a visual materiality of the outer world, in which the outer world and its actions are incorporated by our visual praxis. By way of the visual praxis, the world becomes an extension of our capacities. The material in the world is at once a material out there and a material in here, used as a tool, as a skill, as a memory, incorporated, transformed. As such, the extension of the body goes beyond kinesthetic, material, cognitive or conceptual tools; it is also present in our visual encounter with the world. Be it in visual art or in daily visual scenes, the visual offers us tools, extends our abilities, memory, skills. Being out there, it merges with our position here and implies a disposition, an involvement.
Movement of the viewer: position and disposition
A disposition (…) is the putting to work of a position; it is the discursive deployment through which a position works itself out in practice, that is, in and through the conflicts which it resolves at the same time as it provokes in the situation it finds itself.
Pierre Macherey 1983, p144
The position of the visual relation is embedded in a context and embodied in sensory experience — it is situated. It is objectively defined by other positions, dependent on the actual and potential situation (Bourdieu 1996, p233): a position implies an objective relation or situation in natural and cultural space. Position as such is spatially and semiotically structured by differences, distances, 'op'positions. It entails a syntagmatic position-taking, following the rules of space and relations, perception and physiology, symbols and codes.
Disposition of the visual relation implies the use and/or signification the perceiver makes of the position and of the extension offered by the visual — it is interpreted. It implies a space of possibles, of alternatives and as such is rather paradigmatic. At one side — as a kind of cultural disposition — it refers to the tools the viewer has as a member of a community: the cultural and educational modeling of the human position leaves open an orientation, inside the limits of habit, acquisition and competence. At the other side — as an idiosyncratic disposition — it is singular and implies the subjectivity of the viewer.
We find related conceptions of semiotic disposition in different theoretical views: in psychologically and ontogenetically based theories, in rather evolutionary and phylogenetically oriented views and in the analysis of emergence of semiosis in biological and cultural systems. These theories offer an interesting background to a semiotic disposition of visual praxis.
Kristeva points to the original embodied psycho-semiotic disposition of the human being: ‘We are dealing with disposition that is definitely heterogeneous to meaning but always in sight of it or in either a negative or surplus relationship to it’ (1984, p133). Semiotic disposition here refers to the original meaning-making capacity of the human being, already present before symbolic education and articulation of social codes — a pre-social, psychic capacity (Chapman & Routledge 2005, p169). Semiotic disposition can remain unmarked, part of reflection, or can be expressed along mediating processes and actor centered patterns (Hühn e.a. 2009, p36-7). It is outed in the subjective practice of idiosyncratically renewing or transgressing the order, the symbolic, the structures which are imposed. It de-centers, encounters or resets the 'position'.
From an evolutionary point of view, theorists consider the emergence of meaning as a kind of semiotic disposition. On a basic level it means the ability to represent a small feature of the architecture or topology of the own being and, further on, the formation of structure and meaning out of ephemeral and non-signifying structures. The emergence of semiotic processes in nature and different types of meaning processes in biological systems allowed interaction with an environment and thus life processes and maintenance (Kauffman 1992). There is evidence that emergence of signification and meaning can be found in all levels of evolution (Emmeche 1992, Hoffmeyer 1996, Taborsky 2001).
Broadening these ideas to the phylogenetic origins of humans, we could say that the human being fundamentally needs semiotic disposition for his survival. Our biological position — humans endowed with specific perceptual, motor and cognitive capacities — entails a semiotic disposition, escaping a purely biological, instinctive way of life (Deacon 1999). Steve Mithen proposes the hypothesis of cognitive fluidity (Mithen 1998). Cognitive fluidity is the capacity to integrate ways of thought and items of knowledge previously restricted to isolated cognitive domains: “modern human (…) minds will have a natural propensity to view materials from different perspectives.” (Mithen 2005: 190). They realized “the transition from a domain-speciﬁc to a cognitively ﬂuid mentality” (Mithen 2006, p56). The interaction between different autonomous mental modules allowed different domains as the social, the natural and the technical to become integrated in human's experience and intertwined in human's interpretation and conception. This capacity fulfilled the need to process information about other social beings and their relationships. Cognitive fluidity presupposes three acquisitions: intentionality, symbolic communication and material culture. In the first place, intentionality means that humans can attribute mental states to other human beings and make predictions on the basis of such attributions (Mithen 1998, p170). Secondly, by way of language and representational faculties, cultural knowledge and skills could be shared, exchanged, transformed, leading to a cultural stock which individual endeavor could never attain (Clark 1996, p206). Thirdly, material culture, as an extension of both mental and bodily capacities, increased information storage as well as creative interpretation and practical knowledge that could be used in different contexts by different individuals (Mithen 1998, p184). Such a domain-crossing capacity in which the human being can explore, cross and transform from one domain to another, be it the social, the natural or the technological, enriches ways of being, behaving and meaning-making. In the same sense, we already considered the theory of mirror neurons as offering important scientific evidence of semiotic translations in the brain. Cognitive fluidity has become part of human ontogenetic and cultural development: from birth, humans are surrounded with material and symbolic artifacts resulting from as well as leading to cognitive fluidity.
In semiotics itself, the idea of semiotic disposition or emergence of meaning is inherent in Peirce's thoughts, even if he didn't coin this notion. Semiotic disposition is the basis for semiosis and can be considered as “an interpreter-dependent process that cannot be dissociated from the notion of a situated (and actively distributed) communicational agent.” (Queiroz & Merrell 2009, p130). Peirce starts from the interaction between three poles in the process of meaning-making: the sign or representamen, that stands to somebody for the ground of representation, an object, and that, by way of this, creates in the mind of that person an interpretant (Peirce EP2.478). The world is made up of multiple, different sign systems, but it is by way of a dialectical interaction between an — inner — interpretational process and the world out there, that signification is added (Peirce CP 5.470). Meaning is then an emergent process, realized by the impact of the involved subject in the tryade sign, subject and interpretant.
Semiotic disposition refers here to the role and complexity of the interpretant in the semiosis: the expression of a relation, the formation and transmission of meaning and its re-creation as a further sign by a seeker of meaning. Signification and knowledge acquisition cannot be immediate, but is mediated through signs and interpretants that imply endless and potentially complex processes of semiosis.
Translated towards the dynamics of visual perception and meaning-making we can consider each visual experience as a potential semiotic sign, which possible relational structures of meaning are dependent on its context and situation, implying the relation between sign, object and interpretant. The semiotic process is actualized by way of the interaction between a sign and a subject. For Eco and Peirce, semiosis is an inherently inferential process, that is not regulated by some kind of equivalence rule between expression and content, but necessitates internal reasoning and interpretational processes. As such, semiosis always involves a movement of an interpreter whose “way of acting within the world is either transitorily or permanently changed” (Eco 1995, p149). The process is recurring, makes loops, because a first interpretant becomes the subject — sign— for a next meaning-making round. This leads to unlimited semiosis, a continuous and open process of meaning-making, enhanced and transformed in each new experience, each process of signification being the starting point for another process of signification. These processes are often triggered by leaps or crossings between different senses and experiences, memories and practices, between the personal and the cultural. Peirce's notion of 'abduction' as well as Mithen's cognitive fluidity refer to this kind of embodied and cognitive heuristics humans use in their thought and interpretational processes.
Summarized, the way in which we define here semiotic disposition always involves a meaning-creating subject in a process of performance or signifying practice with an encountered semiotic sign. As with other extensions, position and disposition of the visual praxis often recede into the background of the observer's attention.
Moved by what we see, seeing what moves us
Anybody who interprets a sign brings the baggage of the entire life of the social conventions by means of which he learned what he knows through habituation of his social practices. This includes past experiences and present experiences that collaborate to create expectations regarding what the future holds in store. Sign meaning, then, also integrates other signs and their own interpreters (…).
João Queiroz & Floyd Merrell 2006, p56.
In this (after all) conventional debate between science and subjectivity, I had arrived at this dubious notion: why mightn't there be, somehow, a new science for each object. A mathesis singularis (an no longer universalis)? (…) I wanted to explore it (Photography) not as a question (a theme) but as a wound: I see, I feel, hence I notice, I observe, and I think.
Roland Barthes 1982, p8, p21.
In Camera Lucida (1982), Barthes tries to find a balance between the emotive, subjective viewer and the culturally interested human observer of photography. He considers the image as involving a continuous process of reinterpretation influenced by the position of the image and perceiver and the disposition of the viewer. The position expressed by the historical context of the presentation of the image and by the prevailing biological capacities of the human being, is met by the disposition or performance of the viewer: “suddenly a specific photograph reaches me; it animates me, and I animate it.” (Barthes 1982, p20). Numerous factors outside the frame of the photograph as image influence the productive and performative relation between viewer and image. Moreover, for Barthes, this relation is embodied, or should we say 'lived through' — part of the lived experiences of the viewer. What is seen participates in the different perceptions, in the movements, in the memory and expectations of the viewer — embedded in the body. As such, photographs of landscapes must be 'habitable', not only 'visitable' with the eyes (Barthes 1982, p39-40); a photograph of Barthes' mother cannot reveal her without the recovered experience of the lived relation — “contemplating a photograph in which she is hugging me, a child, against her, I can waken in myself the rumpled softness of the crêpe de Chine and the perfume of her rice powder.” (1982, p65).
Barthes designates two ways of relationship between viewer and viewed, which correspond with our view on disposition. In the first place, a broad culturally based disposition opens the interest of the viewer in a rather conventional way. Barthes names this the 'studium': a general inquiry and commitment of the human being, sustained and shared by education, culture and science.
But in a second stance, a punctum takes place, an idiosyncratic disposition, an absolute singular, unique experience pops up from the subjective viewing experience. Something in the seen has attracted attention, becomes a focus for the viewer. The punctum is the wound or the irruption/interruption in the photograph which coincides with a wound or gap in the experience of the viewer: it implies an impact on the spectator's body, memory, reflections, a profound experience or shock of emptiness or desolation, the cessation of language. The story and history of the photograph merge into the story and history of the viewer, position into disposition, studium disappears in the punctum.
As such, Barthes' Camera Lucida engages into a well informed subjective approach towards a semiosis of viewing a photograph, which leaves the material essence of the sign — the physical, chemical, optical study — out and completes the cultural-contextual essence of the sign and the viewer — aesthetics, history, sociology — with the subjectivity, embodiment and memory of the viewer. It ultimately is the narrative of a personal quest:
at the moment of reaching the essence of Photography in general, I branched off; instead of following the path of a formal ontology (of a Logic), I stopped, keeping with me, like a treasure, my desire or my grief; the anticipated essence of the Photograph could not, in my mind, be separated from the “pathos” of which, from the first glance, it consists. (Barthes 1982, p21).
These (subjective) aspects of a visual praxis remain in general implicit and are articulated rarely; we can find them in reflections on art, in philosophy or in extreme or sensitive 'metaphysical' situations.
Barthes' analysis of the Photograph is specific to the medium of photography, but interesting aspects can be broadened towards other visual experience. What and how we see is dependent on our position in the world and our disposition in culture, appealing to our 'studium' by way of understandable codes — “given cultural meanings that we understand at once” (Barthes 1982, p44): perception, information, representation, signification, surprise, reaction. The studium is in general clear, we can consider it as humans' adherence to what Lotman calls a semiosphere, the framework of semiotic codes and significations prevailing in a specific culture (1990). Visual, and also verbal or auditory, artifacts are part of social sign systems that sustain a shared world. The semiosphere is the established world of sign systems in which the human being enters and is endowed with a quasi natural — though acquired — belief of the existence of objective structures with ontological content. The semiosis implies the processes of mediation and negotiation inside the existing semiosphere. From the point of view of studium, it is the bridge Marco Polo sees — as Calvino explains in the citation of the introduction — whereas that singular old crumbling stone is a possible 'punctum' in the view of the arch. The meaning-searching old man of Borges' tale experiences the punctum just before he dies: his face is the world, or, is the world his face? The punctum, as Barthes explains “is an addition: it is what I add (…) and what is nonetheless already there. (…) The punctum, then, is a kind of subtle beyond.” (1982, p55, p59).
The position and cultural disposition of our visual praxis often hide a subjective disposition, a visual disturbance, neglecting the possible presence of punctum along studium. We are not only seeing, but also 'being there', in a continuous mediation between our situation or position and our subjective disposition. Transformation processes take place from our own standpoint which is itself an evolving motivational, generative and evaluative dynamic. Barthes' “I am the reference of every photograph”, can be translated into 'I am the reference of the viewed, the reference of what I see': my viewing implies “an immediate presence to the world — a co-presence; but this presence is not only of a political order (…) it is also of a metaphysical order.” (Barthes 1982, p84). The seen offers raw material, not a description, nor an interpretation. My presence is needed for semiosis to happen, a semiosis in between viewer and viewed that, by way of this, reveals the essence; and that semiosis will be defined by my position in the world and my disposition towards the co-presence of the seen.
As such, the motion implied in the visual praxis is also an emotion: a bodily way of becoming aware of the impact of and the interactive flow and transaction between viewer and viewed. The word emotion is derived from the Latin 'e' or 'ex', meaning out of, and 'movere', to move. Considered as very intense subjective, private feelings, emotions nevertheless are at the origin of mental and physical responses, influencing and directing continuously our need to make sense of the world:
Emotions are key components of complex processes of assessments, evaluation, and transformation. As such, they are integral to our ability to grasp the meaning of a situation and to act appropriately in response to it. (Johnson 2007, p68).
Emotions are not part of an isolated sphere of human interaction, but are discursive and dialogical ways of making sense of a situation, of the motion in it, and of the feeling and understanding how to move within it. They allow, in a very deep sense, to be 'in touch' with the world and with existing interpretational frameworks, extending, deconstructing them from the viewer's specific relation, disposition, action and reaction. As such visual praxis allows for the complexity of emotion, perception and behavior.
A semiotically inspired analysis of visual praxis will have to move from an analytical view on the relation between sign, object and interpretant, to the process of — subjective — meaning-making itself. The accent here lies on the praxis and the movement — both subjective and shared — in the meaning-making processes of visual experience: the relation between the viewed and the viewer, between emotion and thought, between past and future, between nature and culture. Visual praxis contains a plurality of subjective heterogeneous experiences, linked to memory and world, embedded in socially constructed and shared practices and discourses.
The notion of unlimited semiosis implies the recognition of ongoing mediations between the outer world — the seen — and inner world — the viewer. These are supported by the interpretant. Peirce's theory of the interpretant, as the expression of these relations or the “translation of a sign into another system of signs” (Peirce CP4.127), offers a tryade which semiotically translates the inherent dynamics of visual semiosis. He names these three elements the emotional interpretant, the energetic interpretant and the logical interpretant.
The emotional interpretant is past-oriented, based on recognition and qualitative effect of the sign. It is not only the “feeling produced by it” (Peirce CP5.475) but also the basis for interpretation: “every hypothetic inference involves the formation of such an emotion” (Peirce CP2.643).
Through the mediation of the emotional interpretant, a movement, a concrete effort made in the inner world and directed towards the outer world takes place in the present:
further effect will always involve an effort. I call it the energetic interpretant. The effort may be a muscular one, as it is in the case of the command to ground arms; but it is much more usually an exertion upon the Inner World, a mental effort (Peirce CP5.475).
This energetic interpretant can take potentially infinite variations of mental or physical action.
These interpretants are directed towards action and interaction between our inner world and
the outer world. Further on they lead to a third, a logical interpretant that is self-analysing, and
makes sense of the emotions and physical or mental movements that preceded:
these first logical interpretants stimulate us to various voluntary performances in the inner world. We imagine ourselves in various situations and animated by various motives; and we proceed to trace out the alternative lines of conduct which the conjectures would leave open to us. We are, moreover, led by the same inward activity, to remark different ways in which our conjectures could be slightly modified. The logical interpretant must, therefore, be in a relatively future tense (Peirce CP5.481).
As such, this classification corresponds to the three elements present in the visual praxis: emotion, perception/behavior and interpretation/thought. Moreover, Peirce's tryade offers a possible interpretation for the time-related aspects of visual praxis, linking the past — emotional feeling — with the present — the concrete movement or effort— and the future — generalized rules or habit which regulate “various voluntary performances in the inner world” (Peirce CP5.483). In visual praxis bio-physiological, social and personal operations cross. Its inherent semiosis translates the viewed into concrete signification in and action upon the world.
Our idea of anything is our idea of its sensible effects.
Charles Sanders Peirce, EP1.132.
Peirce stressed the role of interpretation: a sign signifies something for somebody. Sign systems and their tools are not indifferently 'out there', they are 'human' elements of creation and reasoning, both cultural and personal: “means of a culture that we have at our disposal and means that live only in our thinking and acting.” (Hoffmann 2007, p9). As both thinking and acting are an integral part of our human experiences and as human experiences are idiosyncratic events that are diachronically situated in life and memory, seemingly homologous experiences can lead to different, multiple processes of semiosis. No situation of visual praxis has a fixed semiotic preconception, but opens up a semiotic potential — a multiplicity of points of view, interpretations, different articulations — which will/can be actualized depending on background beliefs, culture, expectations and sensory input. Visual praxis encounters different dispositions, cultural as well as personal. When actualized, semiosis always involves processes of interpretation and reflection — or re-interpretation — that reach beyond time and place of the situation. As such, a semiosis takes place between the viewer and the viewed, in an idiosyncratic way, but juxtaposed with broader shared semioses.
This article has explored the complexity of meaning-making processes in visual experiences.
Borges' tale of the face, Calvino's bridge, Klee's moving line or Barthes' photographical experience have illustrated the tensions in visual praxis: those between the subjectivity of the viewer and the shared world of nature and culture, between thought, movement and emotion, between the extension of the body and the meaning-making disposition of the viewer. Visual experience is always a visual praxis, part of and embedded in our embodied and exploratory approach of the world.
Barthes, R. (1982). Camera Lucida – Reflections on Photography (transl. by Richard Howard). New York: Hill and Wang.
Bermúdez, J. L., Marcel,A. J. & Eilan, N. (Eds.) (1995). The body and the self. Cambridge, MA: MIT Press.
Borges, J. L. (2004/1960). Aleph and other stories (transl. Andrew Hurley). University of Michigan: Penguin Books.
Bourdieu, P. (1980). Le sens pratique. Paris: Minuit.
Bourdieu, P. (1996). The Rules of Art: Genesis and Structure of the Literary Field. Cambridge: Polity Press.
Calvert, G.A., Spence, C. & Stein, B.E. (eds.)(2004). The Handbook of Multisensory Processes. Cambridge: MIT Press.
Calvino, I. (1974). Invisible Cities (transl. William Weaver). New York: Harcourt Brace Jovanovich.
Chapman, S. & Routledge, C. (eds.)(2005). Key thinkers in linguistics and the philosophy of language. Oxford: Oxford University Press.
Clark, A. (1997/1999). Being there. Putting brain, body, and world together again. Cambridge: Massachusetts Institute of Technology Press.
Coessens, K., Crispin, D. and Douglas A. (2009). The Artistic Turn: A Manifesto. Leuven: Leuven University Press.
Deacon, T. W. (1998). The Symbolic Species : The Co-Evolution of Language and the Brain. W. W. Norton & Company, Inc.
Eco, U. (1995). The Role of the Reader. Explorations in the Semiotics of Texts. Bloomington: Indiana University Press.
Emmeche, C. (1992). Modeling Life: A Note on the Semiotics of Emergence and Computation in Artificial and Natural Systems. In T. A. Sebeok and J. U. Sebeok (eds.), Biosemiotics. The Semiotic Web. Mouton de Gruyter Publishers, Berlin and New York.
Favareau, D. (2002). Constructing Representema: on the Neurosemiotics of Self and Vision. Semiotics, Evolution, Energy and Development Journal, 2 (4), pp3-24.
Gallese, V. M. D. (2009). Mirror Neurons, Embodied Simulation, and the Neural Basis of Social Identification. Psychoanalytic Dialogues, 19, pp519-536.
Gibson, J. J. (1966). The senses considered as perceptual systems. Boston: Houghton-Mifflin.
Gibson, J. J. (1979). The ecological approach to visual perception. London: Lawrence Erlbaum Association.
Hoffman, M. H. G. (2007/forthcoming). Cognitive Conditions of Diagrammatic Reasoning. Available online as working paper 24 (2007) < accessed 20th of October 2009 >, ed. Georgia Institute of Technology, <http://www.spp.gatech.edu/faculty/workingpapers/wp24.pdf> (Forthcoming in special issue of Semiotica on "Peircean diagrammatical logic," ed. by J. Queiroz and F. Stjernfelt).
Hoffmeyer, J. (1996). Evolutionary Intentionality. In E. Pessa, A. Montesanto, and M. P. Penna eds. The Third European Conference on Systems Science. Rome: Edzioni Kappa.
Hühn, P., Schmid, W. & Schönert J. (2009). Point of View, Perspective, and Focalization: Modeling Mediation in Narrative. Berlin, New York: Walter de Gruyter.
Johnson, M. (2007). The Meaning of the Body – Aesthetics of Human Understanding. Chicago: University of Chicago Press.
Kauffman, S. (1993). The Origins of Order: Self-Organization and Selection in Evolution. Oxford: Oxford University Press.
Klee, P. (1972/ 1953). Pedagogical Sketchbook. New York: Praeger publishers.
Kristeva, J. (1984). Revolution in poetic language (transl. by M.Waller). New York: Columbia University Press, 1984.
Lotman, Yuri (1990). Universe of the Mind (transl. by Ann Shukman). Bloomington: Indiana University Press.
Macherey, P. (1983). In a Materialist Way. In Alan Montefiore (ed.), Philosophy in France today, pp136-154. Cambridge: CUP.
Meltzoff, A. N., & Borton, R. W. (1979). Intermodal matching by human neonates. Nature, 282, 403-404.
Meltzoff, A. N., & Moore, M. K. (1977). Imitation of facial and manual gestures by human neonates. Science, 198, 75-78.
Merleau-Ponty, M. (1945). Phenoménologie de la perception. Paris: Gallimard.
Mithen, S. (1998). A Creative Explosion? Theory of mind, language and the disembodied mind of the Upper Palaeolithic. In Mithen, S. (ed.) Creativity in Human Evolution and Prehistory, pp. 165-191. London: Taylor and Francis.
Mithen, S. (2004). Review Symposium – We have always been … Cyborgs. Metascience 13, pp139–181.
Mithen, S. (2005). Comments V: The Need for an Evolutionary Perspective on the human mind. Archaeometry 47 (1), pp175-207.
Mithen, S. (2006). Ethnobiology and the evolution of the human mind. Journal of the Royal Anthropological Institute, Supplement 1, Vol. 12, pp45-61.
O'Regan, K. J. & Noë, A. (2001). A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences 24, pp939-1031.
Peirce (CP). Collected Papers of Charles Sanders Peirce (= Volumes I-VI, ed. by Charles Hartshorne and Paul Weiss, 1931-1935, Volumes VII-VIII, ed. by Arthur W. Burks, 1958, quotations according to volume and paragraph). Cambridge, Mass.: Harvard UP.
Peirce (EP). The Essential Peirce. Selected Philosophical Writings. Vol. 1 (1867- 1893), Vol. 2 (1893-1913) (= ed. by Nathan Houser and Christian Kloesel, Peirce Edition Project). Bloomington and Indianapolis 1992 +1998: Indiana University Press.
Pineda, J.A. (2008). Mirror Neuron Systems: The Role of Mirroring Processes in Social Cognition. Dordrecht: Springer.
Popper, Karl R. (1972). Objective knowledge - An evolutionary approach. Oxford: Clarendon Press.
Queiroz, J. & Merrell, F. (2006). Semiosis and pragmatism: Toward a dynamic concept of meaning Sign Systems Studies 34 (1), pp 37-65.
Queiroz, J. & Merrell, F. (2009). On Peirce's Pragmatic Notion of Semiosis - A Contribution for the Design of Meaning Machines. Minds and Machines, 19 (1), pp129-143.
Shimada, S., & Hiraki, K. (2006). Infant’s brain responses to live and televised action. Neuroimage, 32(2), pp930–939.
Stjernfelt, F. (2007). Diagrammatology: an investigation on the borderlines of phenomenology. Dordrecht: Springer.
Taborsky E. (2001). The Internal and the External Semiosic Properties of Reality. Semiotics, Evolution, Energy and Development Journal, 1(1), pp4-24.
Warren, W. H. (1984). Perceiving affordances: Visual guidance of stair climbing. Journal of Experimental Psychology: Human Perception and Performance, 10(5), pp683-703.
Zeller, J. (2007). Dynamic Sign Systems in Visual Art. International journal of Philosophy of Culture and Axiology, Issue 7, pp16-24.
E-mail the editors
Pour écrire à la rédaction
© 2010, Applied Semiotics / Sémiotique appliquée