dw_suggestions | Thread Logging (or "Dreamwidth Journal History")

Entry tags:

Thread Logging (or "Dreamwidth Journal History")

Title:
Thread Logging (or "Dreamwidth Journal History")

Area:
Threads / notifications

Summary:
It can be hard to track down all the posts you've made or threads you've started or replied to -- even with tags, commenting in your journal, other people's journals, communities, etc, can make it hard to have any kind of record of your own experiences on the site. If the site can record every one of your posts/top level comments (or the top-level comment you replied to) and report it (somewhere on your console, or into a specific entry in your journal) it will be much easier for people to reread their old posts/threads and track their life on DW.

Description:
Right now it's difficult to find everywhere you've posted or commented to and thus track your 'life' on this site. Basically, if someone wishes to do "thread logging", it all needs to be done manually -- you can track threads which sort of does it a little, but that again is a manual task. There's no automatic record, no recorded history, of where you've commented and what you've said.

Especially as the roleplay community gets larger this is something I feel will be more in demand (as rpers frequently like to reread the character development they go through, but sometimes this can be months or years of posts to wade through -- with an RP like CFUD, you've got six years of roleplay! Or, even if they're not fans of rereading, a lot of sites request you turn in your activity every month, which involves wading back through posts trying to find it, using tags liberally to help yourself keep track (but again, tags are added manually, which means human error). Thread-logging is standard but usually involves manual work.

Perhaps there can be an option on the site which outputs (into a console box or an entry that you set up and somehow mark as a "history" entry) the links for your comments/posts. Obviously doing every comment would be frustrating, so I'm thinking:
A) Any post you make, the post link would go in here.
B) If you make a comment, the History console identifies the top level comment in that thread (whether you started the thread or replied to it) and links it, discarding anything after. (Obviously in the case of B it would also check if it has already linked this thread and not link again if so).

Additional thoughts, details, and concerns:

- It would need to be arranged by date somehow, whether just a chronological order (top-down) or with datestamps of the toplevel comment beside them.

- Since what's just a list of links could become unwieldy fast it would be good for the user to either be able to convert into linktext instead of a plain URL, or at least have a field where the user could add a comment beside the link to identify the content if they choose to (ex: (LINK) - "Umeda and Akiha have a picnic. Mood crab sings them a song.")

- Having a list of the number of your total comments in the thread/post would probably be super useful to RPers (obviously this would mean that either it would have to dynamically update with each reply like the comment section in your profile OR the user could manually run to check for all updates you've made with that username).

- If this WAS going to be planned anyway, obviously the fact that people imported journals would make it difficult to have a 'full' list of their journal's history, unless there were a way to run the function to comb back through all appearances of the relevant username on DW (using their ID) and do a link dump. (This would be ideal for me, as a CFUDer who really wants to reread old threads). Since DW has such a strong search function I think this would theoretically be possible using something similar to the search functionality using the journalname ID, but I'm no programmer.

- Privacy issues -- if in a console you need to be logged into to see this isn't a concern, but if not, whatever journal entry this output into might need to be private by default (with the user able to make it public if they chose to?) in case you didn't want the whole world to see what that journal had been up to.

- This obviously wouldn't track anonymous comments, so if users can manually add extra threads (as in an editable entry) for any 'anon posts' they participate in, or if they did a 'bodyswap' with journals or so on, they could help keep track of these extra things as well. Not totally necessary as long as the journal name's history is added but it'd be a nice bonus for completion's sake.

- If this is impossible to do on the site itself, are there any programmers on the suggestions team or reading this otherwise who may be able to draw up an associated client that could run a similar function of outputting posts/the top comment of threads with a journal name? It's currently difficult to get full thread logs without having to manually keep track of things yourself, which is doubly difficult if you have multiple computers you thread from, if your game doesn't use tags, etc.

Poll #9801 Thread Logging (or "Dreamwidth Journal History")

Open to: Registered Users, detailed results viewable to: All, participants: 58

View Respondents

This suggestion:

View Answers

Should be implemented as-is.
12 (20.7%)

Should be implemented with changes. (please comment)
5 (8.6%)

Shouldn't be implemented.
18 (31.0%)

(I have no opinion)
21 (36.2%)

(Other: please comment)
2 (3.4%)

Flat | Top-Level Comments Only

If this was a feature, I think it should be a paid one -- it seems like something only a fraction of the userbase would use & the amount of coding to make it work might be fairly extensive. (IDK - can someone will more coding experience give an estimate?)

It'd be pretty easy to implement in terms of coding time, but it would be murder on the database load (the Recent Comments page is limited to only the very recent comments because assembling them from various areas of the site is very resource-expensive).

IMO, this is the kind of thing best accomplished by saving email notifications -- paid accounts can turn on the "mail me a copy of my own comments" subscription. I could be convinced, but it'll take a lot of convincing; I don't think the gain is worth the additional load.

This was actually what I was wondering, so thank you for answering my question before I had it. *g* I would think the database load would be more of an issue than anything else, but I like the saving emails idea.

Is there a reason that this would have to be assembled live every time? From the description, it sounds like the user was envisioning something that would be built as a log, with every comment event leaving a link in the log, so it would be a static file that would just need to be accessed, rather than rebuilt every time.

This would be more limited than the original suggestion (no context or security pre-loaded, no history generated from before the feature was put in place), but it might also be a lot more feasible.

Additionally, I would think that this history log should definitely only be visible to the user whose history is logged.

would it be justifiable as a Premium feature?

(deleted comment)

What about as an XML file? Wouldn't that lighten server load somewhat? And it could be subscribed to and accessed just like any other feed on DW, especially once you're logged in? I'm thinking it could be done alternatively as a data/rss or data/atom feed, but with no on-journal/normally-styled equivalent.

Edited (clarity) 2012-03-09 05:51 (UTC)

The load isn't displaying the URLs. The load is finding them in the first place.

Is the database MySQL based?

Surely a command line like:
$query="SELECT comments WHERE username=username ORDER BY username ASC, date ASC, commentURL"; $result=mysql_query($query);
would work?

If it's MySQL, then it's already organized (to an extent) by username (or whatever number is connected to the username and represented across the various tables - because it is the number unique to every person.) Depending on how your tables are set up. The information might ALREADY be in "Username" order and it actually wouldn't be that much of a pull on the database to just dump "these lines" because they're already grouped together. (Again, this is generalized, since I don't know how your back-end is set up.)

As I said below, it could work similar to an "Info Dump Command" and could be set up in queues like the import function is, so that there is more control over the amount of strain put on the database.

From what has been explained to me about the architecture before, the comments are attached to the journals in which they are left, not the comment creator, so that query would need to be for all the journals on all the clusters, increasing the headache factor by quite a bit.

(Thanks, bradfitz.)

(deleted comment)

It's a trade-off -- doing it that way speeds things up considerably for the most frequent use case (loading comments on a single entry) at the expense of making it harder to do things cross-journal.

The code's freely available (the DB schema is in bin/upgrading/update-db-general.pl. The issue is load: it's easy to pull "all your comments in your own journal" (one cluster) and it's easy to pull "all your comments in journal X" (one cluster), but it's hard to pull "all your comments everywhere" (mutiple clusters).

Loading comments is really expensive. To put it into context of the kind of thing we have to worry about: we've been having load issues with entry pages lately on entries with lots of comments, and optimization work on the Comment objects is seeing pretty strong results from shaving off hundredths of a second. A particular query is almost always technically possible, but whether or not it's feasable depends on load calculations; we do not have infinite resources, more's the pity, and database load is the kind of load that can't be solved solely by throwing more machines at it.

Surely it's only murder if you do it in real-time though? What if you had a low-priority background task running on the server (or on another server) to do the logs daily, and append them to a "log" table that's optimised for fast reading? "Your activity history is updated once a day" is not such a ridiculous thing to put at the top of the page.

I wonder if "all threads I've participated in, in X community" would be a good comment subscription type, then -- something between the levels of "all comments to this community ever" (paid community advantage) and "all my comments ever" (paid user advantage).

Not sure how that would do in wear-and-tear, because there would be possibly more back-end checking although at a savings of possibly fewer notifications sent (people who would have subscribed to all community comments subscribing to all my threads instead).

Between saving copies of my comments, the ability to display communities by poster and having enabled search in my own journal I have little problem to find a comment or entry again if I at least vaguely remember what it was about or where I made it. I don't think wading through a massive log file would be any easier.

I made a comment in one of the official comms somewhere that I would love to find again, but it's my problem for not keeping the email notif or for marking/tracking the page. I can think of more than a handful of ways to do that just off the top of my head, with memories, subscriptions, browser bookmarks, web bookmarks, note-to-self posts, tweets, etc.

If it's important to the user to have a record of their comments and activity, I think it should be the user's responsibility to note where that activity goes.

I do want to note that I did say that right now it can be done manually and is (I for one save all my top comments in text files! Except when I miss it, or am threading from a different computer, or otherwise can't access my logs...), but where that's a problem is if you later realize you wanted to do it but didn't do it from the start -- the one thing about manual logging is that it has to be done from the start to have the full value of why you'd do it in the first place, and that the risk of user error is obviously high, with little to do but shrug and get over it when you find holes.

There's definitely no problem leaving this user responsibility -- it's just that a lot of things that can be considered user responsibility are automated to be a nice feature, so I wanted to suggest this as something on that level! (For example, preview to see if your html is working vs checking your brackets manually, spellcheck instead of checking manually, quote button vs quoting manually, etc, even picking a random icon -- these are all things that, if important to the user, can be done manually if there's no option on the site, but these are all options on a comment box because they're nice to haves that were ok to implement).

Edited 2012-03-08 17:32 (UTC)

I mostly log manually myself! But I also thread from my android or have a nasty habit of keeping threads tracked via email... and then having to delete the notifs to make space/accidentally delete the notifs. I tried bookmarking/tabpinning, but I found that doing so only really works for important threads, otherwise you're wading through a list of bookmarks/tabs. But man I'll take any suggestion that doesn't involve me threadhunting for about an hour for an old tag.

I log all my threads and posts manually for the game I roleplay in, which is a matter of minutes tops each day. I think if you want a record of it, it should be your responsibility to make one.

I have to admit I'd really like this, especially if it were done in a post that you could set the permissions on and choose to make public or private or friends-only. I'm way too lazy to do thread tracking, but I'd like being able to go through my threads easily without having to do the extra step of going through gmail's search for each individual thread, and I'd really like to be able to see what other people in my RP are doing without having to worry about missing them, if they choose to make it public. It sounds like it would have to be a paid feature, but eh. Most RPers I know are paid on at least a good percentage of their accounts, so that's not really a problem for me.

Yes, I like the idea of being able to make it public if you are so inclined. With the caveat that if I post a comment somewhere that you don't have access to, you shouldn't see it in the log either, of course... though upthread I suggested creating logs asynchronously and this might not be possible if you were doing that (or might put back the strain on the database that the async logging was meant to alleviate).

I think such a feature could be useful, but not at the cost of the site being usable. Denise said it would be murder on the database and considering many comms are stopping posts at 3k comments to help the site load, it's obvious work would be needed to allow for such a feature, even paid only, and not hinder the site's overall usability.

Once they get the code optimized enough that every comm can easily let posts go to 10k comments if they want to and not cause problems for DW, then I'd highly support a feature like this being added.

Some ways that may help make it more doable, it could be an opt in only feature. Or maybe it could be considered to have a paid package that includes this feature for an small additional cost. Like to give an example, a paid is $3 monthly, paid with this feature could be $4 monthly instead. Doing something like that could allow for features that are neat, but very database intensive and the extra fee could help recoup costs needed to allow these.

So basically, my reason for voting other is that I support this, but I only support it if it won't hinder the site as a whole. At minimum, definitely need the code optimized more.

I'd love this, but like Denise says, it'd be horrible for load times... In the meantime, since you're an RPer there's a sort of workaround that might work for you? This post on LJ explains it; it specifies Gmail but it works with other email providers too -- I'm using Zoho and it works fine for me.

And one thing that post doesn't talk about, since it was written for LJ, is that DW makes it possible to further sort your notifs if you're using the same journal in multiple games, since it puts the journal or community name into the subject line of the email notification.

Thank you for the suggestion! I appreciate your trying to find a workaround for me. Unfortunately, I tried using labels for a while, but it didn't work for the things I'd wanted it to do for a few reasons:

- Too many notifs makes keeping a simple log of start comments a lot harder via gmail, since it tracks EVERY comment you make/get a reply to

- The way threads were stacking (sometimes I get multiple characters stacking together if I was threading more than one character in the same post or subthread) made it kind of messy

- Gmail is really hard to search through on dates etc -- if you're active enough, wading through things in gmail, even when separated by labels, becomes a headache to read whatever you're looking for; also sometimes it mucks up and STOPS stacking things, giving you a million notifs instead of stackable ones...

- Often incomplete (due to not saving notifs until I decided to try thread logging that way since text files have their disadvantages, due to comm importing over, etc) so I can't get a good look at the history. I tried wading back through what I did have saved when I imported over stuff but I've had a few characters with comment counts in the 20,000+ range from history on LJ, and orz.

- Sometimes TOO complete (if I'm tracking a three or four person thread so I can easily know when my turn's up, I get the same notif a couple of times)

I mean, that all said, It's got its advantages and it's definitely better than nothing, and I appreciate the link and advice on how to set it up most effectively! It's not what I'm looking for but it might help others who are looking for something similar and are in this post, too. So super appreciated.

The way threads were stacking (sometimes I get multiple characters stacking together if I was threading more than one character in the same post or subthread) made it kind of messy

Only a small help, but turning off conversation view will stop threads from grouping like that.

Thanks! I'll experiment with that.

imaginemeandyuu, how important is going back to collect every comment ever, versus simply starting fresh when you turn on the feature and going forward?

As has been pointed out here and there, it should be quite easy to automate logging the relevant data (date, url, and subject/first xxx characters, I presume?) to a file or specified entry as you post, and even to verify whether a comment's url already exists in the file/entry. It only gets resource-intensive if you need to go back and collect all the old data from the very beginning.

denise, granted that it's not do-able as a regular paid or premium feature, but could pulling the old data be even remotely considered for a paid extra as a one-time lowest-priority job?

Hi!

Obviously the entire thing is a "nice to have", and "having everything" is nicer to have/more ideal than "having less" in terms of being able to record/track/follow things, but I'm not sure that it's easy to quantify in importance when it's a luxury suggestion, if that makes sense *g*. "Well, I don't NEED it, but it'd be really nice to have, and x would be even nicer to have than y!"

I think on a "thinking of general RPer needs, wanting to reread or be able to easily turn in their character activity", turning it on and going forward from there only is really useful.

For my personal wants, I'm a CFUDER (although not on this journal -- I hadn't even realized I'd put this through on a character journal lol I've been covering my face about that a little). So we imported a huge comm with 7 years of RP in it (and I've been in it since May 2005) in which I can't track down old threads in to read on DW without using the search function and crossing my fingers. Because we only imported in February, all my manually logged links from earlier than Feb of this year are on the LJ comm, and the abandoned LJ comm is hard to read -- we're not on the default because it was causing CFUD users migranes, but the non-default layouts have this quirk where if a thread reaches 150 comments, every thread BELOW that thread abruptly becomes invisible. So if you had just threadlogged with a post instead of an actual thread link, (which I and a bunch of cfuders did for a while because someone had made a program that could track down each username's appearance in posts and spit back out to you the post links, saving some effort), and someone had hit 150 comments above you, the thread you were looking for wouldn't be at all accessible. The threads are still there but you need the direct link to get to them because they won't display on the page. I tried to reread an old character's threads in early January to link for an app I was doing elsewhere, and I think only three or four of his threads were visible from the links I had (though I was able to grab a few more from some gmail notifs I'd been fortunate enough to have saved!).

So from the perspective of the imported RPer, it'd be a lovely thing to have old comment history, as well as going forward (though DW obviously isn't responsible for keeping track of our current or imported activity haha -- it'd just be a nice thing to have if possible). It's not a big deal, though.

Flat | Top-Level Comments Only

Thread Logging (or "Dreamwidth Journal History")

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

+1

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject