![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
foxfirefey) wrote in ![[site community profile]](https://www.dreamwidth.org/img/comm_staff.png){kind=small}
dw_suggestions2009-08-07 02:00 pm
Scheduled Imports
Title:
Scheduled Imports
Area:
importer
Summary:
I want to be able to schedule regular imports of other journals.
Description:
A lot of people maintain journals in two places, like on DW and LJ. And maybe they make posts on LJ that they want imported instead of using the crossposter, but don't want to have to manually do it every now and then. Or they want to let people comment in both places and automatically get new comments imported into their DW. It would be really useful if they could set up automatic scheduled imports so that this would happen daily, weekly, biweekly or monthly.
This should probably be implemented after the crossposter and importer talk to each other. It would also require stored account details like service, username, and password.
Because this encourages heavier use of the importer, which is undesirable to overload, and because free users can still use the importer whenever they want, this could easily be a paid feature. Additionally, it could be tiered by minimum time between imports, where Premium Paid accounts could perhaps set up daily imports and Paid accounts could have weekly imports. Another tier could be number of accounts allowed to have scheduled imports. Automatic imports for regular backups along with search would make a paid DW a good second account for heavy users of LJ, whether or not they're actually into using DW as a place to read.
Downsides to this: it could clog up the importer. We don't want any one service we're importing from to be burdened by our imports.
This suggestion:
Should be implemented as-is.
18 (46.2%)
Should be implemented with changes.
9 (23.1%)
Shouldn't be implemented.
0 (0.0%)
(I have no opinion)
12 (30.8%)
(Other: please comment)
0 (0.0%)
no subject
I like this. Good compromise between database-load and what is a useful feature.
no subject
In addition – looking at it in terms of usefulness, if the importer is scheduled daily, is that actually going to import all comments? I know that I comment on posts that are days old, if it's been days since they've been posted before I see them; I'm sure I'm not the only one. I don't know if the importer checks old, already-imported entries for new comments, but if it doesn't, scheduling for daily import seems like it would be a bad idea.
Also, I hope that the daily/weekly or weekly/monthly import schedules being tossed around in this suggestion and discussion so far are minimums, rather than the only option available. I could have a paid premium account and want to import monthly, for instance, instead of daily or weekly. Is that automatically considered to be an option?
no subject
no subject
no subject
It's planned. At this point, it creates a duplicate entry. That's part of what was meant by delaying until the crossposter and the importer talking to each other.
no subject
no subject
no subject
But yes, the importer will import all comments since the time of the last import, as the comment imports are actually separate items from entry imports. So even if somebody makes a new comment on a really old entry, it'd still get imported.
And yes, I think the person should be able to choose which time span, up to a minimum. Some people won't want it to be that frequent, and less frequent is better anyway for load.
no subject
If you have X number of imports that you need to accomplish in this 24 hour period, or this 7 day period, you may be able to take a look at traffic and load, and say "Okay, this is just 6.5 days for this user, but if we put a bunch of imports here, that will fit nicely into that low-load time and free up more resources for when they speed up half a day later." And so forth.
As a user with scheduled imports, I would want to be able to check when it last ran, and when it was scheduled to run, somewhere. My sanity does not depend on it being precisely the same time so long as I can see when it's upcoming. (Other people may not feel the same, but it is not as if the current import-now importer has a precise finishing time even when you know when you started it.)
no subject
no subject
no subject