My apologies to those out in blogland who might not care about these sorts of interchanges -- it is very difficult to spread this kind of content in tiny little nuggets. the posts are just, well, long.
From elw@stderr.org Tue Jun 29 22:18:39 2004
Date: Tue, 29 Jun 2004 22:17:30 -0500 (CDT)
From: elijah wright 
To: Elizabeth Lawley 
Subject: Re: re many2many post on blog research


> Elijah-
>
> Thanks for your note.

most welcome.  I really do want to see some of this stuff engaged with in
a very deep way.


> > I was happy to see this work being presented, as well, though not so
> > uniformly as you seem to have been.  As a gloss of what I saw
> > happening in that panel - I'd probably categorize most of it as
> > qualitative and mildly sociological.  Alex's presentation was
> > explicitly so, while others (Thomas Burg) tended toward an approach
> > grounded in critical theory that is somewhat less common among
> > AoIR-ites.
>
> You're right that most were qualitative in nature--though Cam Marlow's
> was explicitly quantitative, based on his blogdex data. While I'm not at
> all against quantitative approaches, I think it's more difficult to
> apply them usefully before we understand the nature of what we're
> studying--and I don't think most of us do at this point. Qualitative
> research allows us to identify the characteristics and variables, while
> quantitative allows us then to measure and test them. And on a macro
> level, quantitative data is allowing us to make out the contours of the
> landscape--Cam's work is one example
> (http://overstated.net/04/05/24-weblogs-and-authority.asp), while the
> work they're doing at Technorati is useful as well
> (http://www.oreillynet.com/pub/wlg/4411).

I tend to agree that a balance between qual and quant is useful, but I
suspect that I may be coming at that from a slightly different direction
than you are.  =)  We all have our individual (and peculiar) trajectories,
I think.


> > My notion of what ought to be going on, here - and i think this is a
> > key practice that needs attention from the blog-scholarship group as a
> > whole - is that more energy should spent on critique and commentary
> > when our friends, or people that we agree with, are the ones talking.
>
> Point well taken. To be fair, a lot of that has taken place on the
> weblogs of those scholars--when Alex, or Thomas, or Cam post about what
> they're working on, they tend to get comments and pushback from their
> readers. But it's true there could be more.

Unfortunately, that's not peer review.  I wish that it were.  And
aggressive, critically conscious (but still empathic!) peer review makes
for far better research.


> > I would probably not have mentioned this, were I you, but it does
> > raise an interesting axis of discussion that cries out for attention:
> > how can we better integrate what I'll call "experience papers"
> > (scholarly papers written by folks who define themselves publicly as
> > bloggers first, and academics second) with papers that are written by
> > people who identify as "scholar first, blogger second or not at all"?
>
> I find that a troubling approach. I'm not "one first, the other second,"
> nor would I characterize others in that way. No more than I'd say I was
> "woman first, scholar second," or "American first, scholar second." (And
> in the former example, I think that it's also true that women writing
> about gender issues tend to have a little more credibility in that
> space, and that men writing about gender issues have to work a little
> harder to prove that they're genuinely attuned to and aware of the
> issues.)


If you have an alternate ontology that describes the ways in which
academic bloggers and blog researchers are different, I'd love to hear it.
It clearly isn't so simple as "whether the person blogs or not" - *most*
of the people taking stances that you're labeling as empirical actually
*are* bloggers of one shade or another.

These kinds of orderings are always a little bit problematic, and I think
that they should certainly be open to debate.



> My concern is that trying to describe, categorize, measure, or analyze
> something that you haven't spent a good bit of time "within" is, to me,
> problematic. It's like trying to do cultural studies without actually
> spending some time in the culture you're studying.


This opens a deep philosophical can of worms.  If we accept (as I think we
do) that research on weblogs is actually still in its rather early days,
then *how* do new scholars have the experience to study the topic?  This
is like asking for an employee with 25 years of java programming
experience when the language is only 11 years old.

Weblogs are not really so new, anymore.  Many of us have had four, five,
or six years to think about the mechanics and the demands of the form.
Claiming otherwise is rather disingenuous, but still we are seeing
scholars who start their conference presentations as "no one has really
studied blogs before"...  I can only conclude that this is bad
scholarship, rather than assuming that weblogs are merely seldom studied.


> > I'm not sure that it is fair to critique panel presentations based on
> > the presenter's apparent experience (or lack thereof) with weblogs;
> > essentially, you are forced to rely on your perceptions of their
> > knowledge rather than their *actual* knowledge.
>
> Actually, my critique was based on what I saw as flawed methodology and
> problematic solutions, which I then attributed to lack of experience
> with the medium. Trying to characterize several million weblogs
> worldwide (technorati currently tracks 2.8 million) based on two small
> samples, is, I still believe, not a viable approach.


For initial, rough-cut research that's not been done before, random
sampling methods are remarkably accurate.  We expended an enormous amount
of energy in ensuring that we were establishing a statistically random
sample, and on discussing the various quirks of our methodology.  That
should not be discounted - folks who were involved in that paper have a
rather deep understanding of what we did, why, and what the consequences
were.  Those characteristics of our data are quite a bit clearer in our
paper for HICSS than in the presentation that we did for AoIR.


Large-scale sampling (where you run analyses across several hundred
thousand weblogs, for example) has its own flaws.  Dirty data, of course,
being one of the most difficult to surmount.


> > We didn't participate in Alex's mega-blog-panels at AoIR because our
> > work simply *didn't fit* with what was being done in the panels. I'm
> > not sure how we (other presenters) should take your critique,
> > actually, or even whether you're talking about us.  (I suspect so,
> > based on what our core material was.)
>
> There were a number of weblog-related presentations at AoIR--it wasn't
> just yours I was referring to. However, if you'd presented as a part of
> Alex's panel, I suspect I would have had the same reactions. It wasn't
> an issue of being "inside the circle."


Unfortunately, other researchers in the community become suspicious when
you mention an entire panel in a glowing light.  I was in the audience at
that panel; I remember quite clearly what the flaws and issues were of
each paper.  It often isn't polite for me to address those issues
directly.


> > So where do content analytic paradigms and genre analysis fit into all
> > of this?  They're canonical research tools and can't easily be
> > dismissed.
>
> I'm not dismissing them, and I agree that they're valuable tools. But I
> think they can only successfully be applied once we have a better
> understanding of what the genre (or genres) we're looking at are. A
> random sampling that excluded sites based on controversial criteria
> (excluding livejournal, for example),

At the time, the LJ site (to anthropomorphize) wasn't willing to refer to
itself as a site for weblogs.  Soon after our research was done, that
changed.  Were we to do the same research now, I think that we would
include LiveJournal and Diaryland as well.

In that moment, though, it was inappropriate to include those two sites in
our sample.

Interestingly, if you'll remember, including LJ and Diaryland in our data
would have made the patterns in our data even more striking.  Including
them enhances our results, rather than calling them into question.  :)


> or categorizes them based on only two weeks of posts (in any given
> two-week period it would be easy to categorize my blog in a way that
> would not be representative of it over the longer time frame) is not, I
> think, a good starting point for content analysis.

Random sampling accounts nicely for this.  If we happened to
mischaracterize a particular weblog, it is likely that we would have a
good chance of also mischaracterizing another one.  The effects should
cancel each other.


> > Not everyone is an ethnographer, nor should we all attempt to be.
> > For that matter, desires that everyone be deeply immersed in the
> > activity of blogging compromise the community by removing objectivity
> > and empirical research from the realm of possibility.
>
> I'm not just talking about ethnography. I'm talking about fundamental
> misunderstandings of the medium caused by overly superficial
> assessments.


I'll say this again, as a kindness:  How, exactly, is it that you know
that other people's assessments are superficial?  You, like the rest of
us, are far from omniscient.  This kind of attitude toward other
researchers is entirely unproductive, and leads to a lot of very hard
feelings.  This kind of assessment is especially difficult in a forum like
AoIR, where many of the researchers presenting their work do so using
techniques and methods that not everyone in the audience understands.


> I only mentioned ethnography as one of five aspects of
> blog research that I think have some potential for exploration. But
> just as it's difficult to do literary theory if you don't read
> literature, and difficult to do meaningful analyses of Japanese culture
> if you haven't spent some time in Japan, and

I mentioned ethnography as a means of describing the general kind of
participant-researcher work that (overall) you seem to be advocating.


> difficult to do accurate assessments of computer use if you don't know
> the terminology associated with computers, I think it's very difficult
> to do research into blogs without a solid understanding of the form, one
> that's particularly difficult to obtain from the outside looking in. Not
> impossible, certainly. But as I said above, the issue was what I saw as
> problems with the methodology and conclusions, which I simply inferred
> were based on misunderstandings born of unfamiliarity.


See above.


> > And, as far as I can tell, no one has suggested doing so.
>
> Perhaps not. But generalizations based on random samples that do not
> acknowledge the differing impacts of weblogs on the communities within
> which they function seem to me to be generalizing.

Yes, and intentionally so.  That aspect (statistical or demographic
analyses of what's present) is valued for different reasons and for very
different purposes than discrete analyses of how individual weblogs
interact with each other.

I look forward to the day when have established
terms and understandings as a community - right now the whole research
community is busy importing disciplinary baggage as a means of having
meaning-making tools that can encompass weblogs.


> Kevin Marks of
> Technorati has made an interesting point, which is that the
> most-linked-to weblogs still have only thousands of inbound links, a
> miniscule fraction of the total number of blogs.

Yes - that link distribution is actually quite interesting.  Hard to draw
conclusions from, but very neat.

> There really isn't "an A-List," outside of specific areas. There's a
> political "A-List," a techno-geek "A-List," etc. But if you were to ask
> the vast majority of weblog readers and writers to name ten blogs out of
> the top 100, the vast majority couldn't do it. The impact of blogs on
> readers happens in smaller, interactive circles, not mass media
> broadcast modes.

You will probably enjoy our paper for HICSS-38.  Hopefully it will be
accepted :)  Some of these issues are directly addressed therein.


> I do wish you'd posted your questions and challenges as a comment to the
> blog entry, so that more people could participate in this conversation;
> maybe you'd consider doing that at some point? Or posting it to your
> blogninja site so that I could point to it?

Probably simpler for me to post it to my personal blog -
www.geek-guides.com.  How about I post the original message there, you
post your response where you wish, and then i post an (edited) copy of
this one as a followup entry?

[the blogninja site doesn't have any sort of trackback or pingback
facility on it - it needs some TLC that I haven't had time to give it.  my
personal blog also needs TLC, but at least it is functional these days ;)]

[[I say edited because I think I really was a bit too harsh in response to
you in one paragraph above - the one that has the "How, exactly, is it
that you know that..." content.  I genuinely want you to intraject the
reasons why I might have taken that aspect of your original post poorly,
in hope that you find it somewhat transformative.]]

If you don't want me to edit, I won't.  I'm fine with it either way.

I'll drop you another note with a link to a blog post of the original
email in a minute.


> I've found that open
> conversations on blogs tend to facilitate interesting emergent ideas, as
> well as providing an opportunity for serendipitous additions of ideas
> from people we might not have thought to ask.

Absolutely - and those are some of the people that I'd like to see a *lot*
more text from.  The gang of us who're interested enough to actually
*work* on weblogs are not necessarily the most skilled or appropriate
group - but there's some joy in it, I think, that keeps us going.


> Like you, I'm hoping that there can be some collaboration and
> cooperation among researchers in this space. That was my primary goal in
> posting what I did...to create a public discussion of blog research
> issues that would spark some discussion.

it seems to have worked :) :) :)


> And I do apologize if you found what I wrote disrespectful--it
> certainly wasn't intended as such.
>
> best,
> Liz

For a while, but I've pretty much gotten over it now.  And I'm seeing what
you wrote in quite a different way than I originally did.  Getting a
thoughtful, engaged email response certainly has helped *my* attitude a
lot...

--elijah
writebacks...
trackback
TrackBack ping me at:
http://www.geek-guides.com/ell2.writeback.trackback
comment...

Name:
URL/Email:	[http://... or mailto:you@wherever] (optional)
Title:	(optional)
Comments:
Save my Name and URL/Email for next time