• first brainstorm sesh
• second brainstorm sesh

4.5 pages = 1250 words
12 pages = 3,333 words

research question

This project offers researchers a new method for the study culture -- not from experiments or surveys, but from natural, unguided conversation. This method is the ultimate fly-on-the-wall ethnographic account, where the ethnographer goes entirely unnoticed and takes perfect notes. Even more amazing, researchers can release all their ethnographic data with no privacy concerns whatsoever. The "flies" on my proverbial "wall" are skin conductance sensors, attached to the index and middle finger of each person, which measure their emotional state, roughly speaking. Synchrony in emotional reaction indicates a common understanding between the synchronous individuals, even more so when many individuals are synchronous. I identify moments of broad synchrony in small-group conversations, so-called "meaningful moments," and use them to place participants culturally. That is, over time these synchronies sketch the outlines of the subcultures of sentiment and understanding underlying those who are participating in conversation.

The method is inexpensive, is generalizable across language and culture, and involves no understanding or interpretation of the signs and symbols of the culture studied. Thus the method would enable a vast amount of data-generation, and any methods developed for analysing this data would be entirely agnostic to the context being studied. In this way the method promises unique opportunities for cross-cultural studies in cultural organization.

Before I dive into the literature review I will say a bit more about the skin conductance sensor which lies at the heart of this study. A word of caution, this description will be very rough, and brushes over a lot of complexities which Boucsein (2012) spends almost 600 pages discussing. That being said, the conductance of your skin is measured by sending a small amount of electricity from one electrode to another through the skin, and inspecting this electric current. Changes in these electrical properties over short time scales index the activity of the sweat glands between the two electrodes. Unlike the rest of the body, the sweat glands in a person's hands and feet do not respond to the body overheating (Kerassidis 1994). The eccrine sweat glands are instead influenced by the sympathetic nervous system, the fight-or-flight system in the brain (Boucsein 2012, Ch. 1.3). So the (huge) up-side is that these cheap little sensors can pick up fear and anger, glee and worry, in real time. The (also huge) down-side is that they say little about which of these occurred. Skin conductance only indicates magnitude, and even then there can be a lag between stimulus and response of up to three seconds. Despite the limitations, we take advantage of this inexpensive pulse on the emotional brain.

literature

Using synchronization as an indicator of meaning has a rich theoretical and empirical history. (YOU HAVE NOTHING HERE. WHY DID YOU GO HERE?) Theoretically, it echos Mead's theory of the development of language, of symbol, and of understanding through the internalizing the response of the other (Mead). Interaction ritual chains are another theoretical concept focusing on synchronization, in this case the synchronization of "moves". When these moves also entail some intensification or excitement this is the source of an interpersonal connection between individuals (as described in McFarland et al. 2013).

Following Schutz (1967 [1932]) and more famously the elaboration and experiments of Garfinkel (1967), people get uncomfortable when their "taken-for-granted" world breaks down. That is, when other people or their environments do not behave according to their understanding of the world. Garfinkel operationalized this concept of Schutz by asking his students to break minor social conventions, or typical understandings. For example, students might be instructed to enter their home with the mindset that they were only visiting. They would ask if they could use a cup, immediately rinse the dishes instead of leaving them in the sink, and act like the didn't know where their room was. These homework exercizes were surprisingly hard for students to follow through with, and were successful in making their friends or family quite irate.

A growing empirical literature buttresses these theoretical insights, as well as clarifying and expanding them. When telling a story, there is a detectable synchrony between the brains of the storyteller and the listener, the magnitude of which is moderated by how much the listener understood what was being said (Stephens et al. 2010). Recently there has been a huge growth in literature assessing the success of psychotherapy () and in understanding and diagnosing autism () through measurement of physiological coupling.

(FINISH THIS)

methods

I plan to recruit participants from a small community, hanging flyers and announcing the study through local media and events. The announcements will contain a link to sign up for the study, which will ask participants to commit to four weekend days they attend the four-hour study (lunch included). Once a day’s roster has filled with 30 participants, I close signups for that day.

When the participants arrive, I will speak to them all at once, describing the study, answering questions, and having them fill out consent forms. I will instruct them on conversational etiquette, most importantly to make sure everyone gets a chance to speak, that there should only be one person speaking at a time, and others should listen to the person who is speaking. I pass out skin conductance response sensors and microphones and demonstrate how to properly install them on their fingers and lapel, respectively. These skin-conductance and audio data are recorded by a small device in their pocket. This data collection module also sends diagnostic information regularly so researchers can quickly troubleshoot device malfunctions. Once a person’s sensors are on and working, they are encouraged to brainstorm conversation-starters on the chalkboard or have some coffee and snacks. This whole process takes at most one hour.

Each person is then given their table schedule, telling them where they will converse. They are told to go to their table and strike up a conversation. The tables are circular, and small enough to ensure everyone can be seen and heard by others. Each table is equipped with an 180-degree camera which records continuously throughout the study. The conversation ends after 12 minutes, and the next conversation starts in 3 minutes, amounting to four conversations per hour. Conversation transitions are signalled by two bells, and enforced by the researcher. Participants continue like this for 2.5 hours (10 conversations in total). The participants can then remove their sensors, are then served lunch, and possibly entertained in some way.

While the participants eat, an algorithm goes to work on the data which was collected, preparing for Phase 2. This algorithm looks for “meaningful moments,” five-second segments where more than two individuals have a similar pattern of skin conductance. Boker et al. (2002) has developed the statistical machinery for doing this, the so-called "windowed cross-correlation, peak picking method" (WCCPPM). WCCPPM gives a measure of how much two individuals coordinate, and when they are most in-sync. It also identifies the lag between series which maximizes their correlations. This gives some indication of the ordering of reactions. Meaningful moments can then be defined as more than one of these cross-correlations being large at the same time. For each meaningful moment which is identified, the algorithm creates a 15-second video clip, starting 10 seconds before the moment, and ending 5 seconds after. These videos will be used in Phase 2.

Participants are instructed to wash and dry their hands and replace their sensors before starting Phase 2. Phase 2 lasts a half hour, and asks respondents to respond to these moments. They are asked to simply respond verbally to the video, stating what they think is going on in that moment while they rate the extent to which they understood what is being said, and the extent to which they understand it now. They are given 15 seconds to respond before the next video clip starts. Each person is presented with 45 moments. These moments will be partly moments they were part of (15), moments in which they were present but not synchronous (15), and moments which are totally new to them. These are organized into blocks and labeled by their type, to help reduce cognitive load. Finally, participants are asked to fill out personality surveys and to list their relationships with any others participating in the study. They are also presented faces for each person in each conversation and are asked to rate on a Likert scale + "don't know": how much they have in common, to what extent they feel they are similar to this person, and whether they would be interested in having another conversation with this person in the future. Participants are paid $100 for their time and energy.

analysis

Consider the bipartite graph between individuals and meaningful moments, where an edge exists if the individual participated in the moment, and is absent if they did not. This graph changes depending on the lower-bound of correlation with others which I define as "participating."This parameter should be varied in robustness checks, but a good start for this analysis is to consider the top N correlations, where N defines the number of edges I want in the bipartite graph. This N can be tuned until the graph attains some nontrivial structure, and possibly depending on computational constraints. I must also be careful in analyzing this structure to acknowledge the many missing values, where an individual did not witness the moment, and thus I do not know whether they would have if they did.

This graph structure between individual and moment can be analyzed with network analytic methods. An initial goal would be to make broad partitions of the members and events based on their connections. Doreian et al. (2003) introduced bipartite blockmodeling (biSBM) for exactly this purpose (and the stochastic blockmodeling of Karrer and Newman, 2011, likely will be of use). This method groups columns and rows simultaneously, and generates a graph structure for the connection between these groups of, in our case, individuals and situations. Blockmodeling attempts to minimize an objective function which measures how frequent contradictory ties occur. There are other methods with the same output which can be evaluated as well, for example the excellent information-theoretic co-clustering of Dhillon et al. (2003). These methods are all alike in that they consider moments as similar to the extent that similar people share them, and people similar to the extent that they share similar moments. A more sophisticated approach could be obtained in future studies by specifying a ???

Once clusters are identified in this way, I examine the moments and individuals which are grouped together qualitatively, looking back to individual responses to situations, the content of the situation itself, and the surveyed attributes of the people who are grouped. The goal at this stage is to tune this method to index cultural patterns broadly. Upon validation of this method, we can proceed to more sophisticated analyses of the cultural categories thus derived.

Two derivative graphs of immediate interest are the projections onto participants and moments, which create the participant-coupling network and the moment-coupling network.

Attention can be assessed through a comparison of skin conductance responses from people's membership in the moment and their retrospective viewing of it. If they exhibit a synchronous reaction upon rewatching whereas they did not when they were present, we can infer the original lack of response was due to a lack of attention. This can be corroborated against their self-reporting of attention. Another index of attention could be derived from computational video analysis, assessing simply whether they are looking at the individual who is speaking, although this measure is not definitive.

expectations / results

Prior work on skin conductance synchrony suggests multiple expectations for the data thus collected, and can act to validate the methods here developed. The average synchrony between each pair of participants across all conversations should predict their ranking of affinities for each alter () and synchrony in individual situations should predict comprehension, affect, and attention to the situation ().

The more novel analysis lies in the clustering of individuals and of events based on their co-occurrence. Broadly speaking, I expect the bipartite person-event network to indicate an underlying cultural structure. This can only be validated at this stage through comparing a qualitative understanding of the individuals and moments to the results of the clustering analysis of this graph.

This study has only been a proof of concept, a demonstration in the ability to collect cultural data via synchronization in skin conductances between individuals. I encourage those reading to give this method a try, or to analyze the data I've collected. All behavioral and physiological data from this study are publicly available, along with the tools I used in analyzing it. This is in the hope that researchers can easily pick up where I have left off, exploring the data, possibly even developing more successful and more precise methods for deriving culture from situational membership.

discussion

I have outlined in this paper a new method for mapping common understandings of a group through the non-invasive measurement of psycho-physiological indicators during naturalistic social interaction. Prior research has suggested the power of this approach at tapping into the often subconscious social dance we partake in daily. In applying this method to mapping commonalities between individuals and between situations prior to any understanding by the researcher of what was being said, I suggest a wedding of the benefits of qualitative and quantitative analysis. The qualitative is in the correspondence between physiological synchrony and common meaning. The quantitative is in the ability to collect large amounts of data, and compare easily across cultures, languages, and contexts.

I see a few routes to expand and deepen this study. First, I could measure skin conductance at multiple sites for each person. It has been agreed that different parts of one's hands and feet have a different set of communication lines with the ANS (Boucsein 2012, 1.3.2; Picard et al. 2016). On a related note, the design and data would be made stronger by finding or becoming an expert in electro-dermal activity. A quick skim of "Electrodermal Activity" (Boucsein) will confirm that it's not as simple as measuring the amount of sweat. For example, researchers have theorized that the amount of sweat cannot actually change as rapidly as EDA does, and conjectured these changes in conductance indicate a change in the permeability of the sweat glands (Boucsein 2012, 1.4.2.2).

references

• Boker, S. M., Rotondo, J. L., Xu, M., & King, K. (2002). Windowed cross-correlation and peak picking for the analysis of variability in the association between behavioral time series. Psychological Methods, 7(3), 338–355. https://doi.org/10.1037/1082-989X.7.3.338
• Boucsein, W. (2012). Electrodermal Activity. https://doi.org/10.1007/978-1-4614-1126-0
• Delaherche, E., Chetouani, M., Mahdhaoui, A., Saint-Georges, C., Viaux, S., & Cohen, D. (2012). Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines. IEEE Transactions on Affective Computing, 3(3), 349–365. https://doi.org/10.1109/T-AFFC.2012.12
• Dhillon, I. S., Mallela, S., & Modha, D. S. (2003). Information-theoretic co-clustering. Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD ’03, 89. https://doi.org/10.1145/956750.956764
• Doreian, P., Batagelj, V., & Ferligoj, A. (2004). Generalized blockmodeling of two-mode network data. Social Networks, 26(1), 29–53. https://doi.org/10.1016/j.socnet.2004.01.002
• Garfinkel, H. (1967). Studies in Ethnomethodology. Englewood Hills, New Jersey: Prentice Hall, Inc.
• Hasson, U., Ghazanfar, A. A., Galantucci, B., Garrod, S., & Keysers, C. (2012). Brain-to-brain coupling: a mechanism for creating and sharing a social world. Trends in Cognitive Sciences, 16(2), 114–121. https://doi.org/10.1016/j.tics.2011.12.007
• Karrer, B., & Newman, M. E. J. (2011). Stochastic blockmodels and community structure in networks. Physical Review E, 83(1), 016107. https://doi.org/10.1103/PhysRevE.83.016107
• Kerassidis, S. (1994). Is palmar and plantar sweating thermoregulatory? Acta Physiologica Scandinavica, 152(3), 259–263. https://doi.org/10.1111/j.1748-1716.1994.tb09805.x
• Kupper, Z., Ramseyer, F., Hoffmann, H., Kalbermatten, S., & Tschacher, W. (2010). Video-based quantification of body movement during social interaction indicates the severity of negative symptoms in patients with schizophrenia. Schizophrenia Research, 121(1), 90–100. https://doi.org/https://doi.org/10.1016/j.schres.2010.03.032
• McFarland, D. A., Jurafsky, D., & Rawlings, C. (2013). Making the Connection: Social Bonding in Courtship Situations. American Journal of Sociology, 118(6), 1596–1649. https://doi.org/10.1086/670240
• Picard, R. W., Fedor, S., & Ayzenberg, Y. (2016). Multiple Arousal Theory and Daily-Life Electrodermal Activity Asymmetry. Emotion Review, 8(1), 62–75. https://doi.org/10.1177/1754073914565517
• Prochazkova, E., Sjak-Shie, E. E., Behrens, F., Lindh, D., & Kret, M. E. (2019). The choreography of human attraction: physiological synchrony in a blind date setting. BioRxiv. https://doi.org/10.1101/748707
• Ramseyer, F., & Tschacher, W. (2011). Nonverbal synchrony in psychotherapy: Coordinated body movement reflects relationship quality and outcome. Journal of Consulting and Clinical Psychology, Vol. 79, pp. 284–295. https://doi.org/10.1037/a0023419
• Rogers, K. B., & Robinson, D. T. (2014). Measuring Affect and Emotions. In J. E. Stets & J. H. Turner (Eds.), Handbook of the Sociology of Emotions: Volume II (pp. 283–303). https://doi.org/10.1007/978-94-017-9130-4_14
• Schütz, A. (1967). The Phenomenology of the Social World. (F. L. G. Walsh, Trans.). Northwestern University Press.
• Stephens, G. J., Silbert, L. J., & Hasson, U. (2010). Speaker-listener neural coupling underlies successful communication. Proceedings of the National Academy of Sciences of the United States of America, 107(32), 14425–14430. https://doi.org/10.1073/pnas.1008662107
• Vinciarelli, A., Pantic, M., & Bourlard, H. (2009). Social signal processing: Survey of an emerging domain. Image and Vision Computing, 27(12), 1743–1759. https://doi.org/https://doi.org/10.1016/j.imavis.2008.11.007

appendix: budget

appendix: video & analysis