afuna_archive

Not the inside-inside, Perl guts, but trying to update all (DW) layout layers, to get all the updated features!

Long-term goal (actually, medium-term, because once the site goes live, everything needs to be stable, so I won't be able to do any drastic changes) is to turn this layout/feature chart entirely green. I know it's not possible; I can at least try what is reasonable.

One The hard thing about developing with S2 "officially" is that the quickest (only?) way to get feedback on your changes is to apply the layout to your journal, use the S2 compiler (layeredit.bml), apply changes, check journal, rinse repeat. However, then you have to copy the updated layer to bin/upgrading/s2layers. I've found that last step error-prone: tedious to pick out the changes bit by bit and transfer them manually. If I try to replace the entire layer in one go, I find that I have a hard time copy-pasting more than a screenful in vi (hm, have I ever tried just truncating the file and then pasting everything into a blank file? I don't think I have. Maybe I should do have tried that before trying to work around it).

So, anyway! Perl script! You give it the name of the layout. It reads the updated version which is currently "live" (on your dev site need to set the base URL; localhost works fine reads from ljconfig.pl). It then updates the static file in bin/upgrading/s2layers. After that, just need to do cvsreport.pl or hg diff or svn diff or commit or whatever. I've stuck in my ~/bin, so I can run it from anywhere, which is pretty convenient:

http://afunamatata.com/scripts/dreamwidth/s2sync.old

(This assumes that $LJHOME is set properly, and live versus cvs directories are in their normal LJ-setup locations, though this is really more for DW, since it overwrites bin/upgrading in DW-free)

( Sample in action )

At some point, I may find it useful to modify it so that it accepts and upgrades multiple layouts (and themes??), but too much trouble to do so now, since I'm not touching themes at all.

And hopefully this will make development that much easier :) I was dreading keeping my live layout versus my diff-able layout in sync, but I am looking forward to coding this weekend now.


Update
exor674 just came up with a much prettier version <3 http://pastebin.com/f2fc673b3

I'm appropriating it for my own use.

Update2
Slightly tweaked version here: http://afunamatata.com/scripts/dreamwidth/s2sync!

(Basically, sanity check to make sure it's a layout layer, and also put the output directory into a variable, so that people who want to stick it straight into live instead of under cvs/ can change just the one line)

It now accepts either layer id (as a number), or layout name or redist_uniq, so either of these will work:
$ s2sync 333
$ s2sync bob/layout
$ s2sync bob

Also, output slightly changed:

dh-afuna@li-166:~/dw/cvs/dw-free$ s2sync 336
Can only handle layout layers at /dreamhack/8080-afuna/bin/s2sync line 33.
dh-afuna@li-166:~/dw/cvs/dw-free$ s2sync 333
bob/layout written to /dreamhack/8080-afuna/dw/cvs/dw-free/bin/upgrading/s2layers/bob/layout.s2


I forgot that certain characters under certain conditions will cause parts of a word to be expanded as a variable in Perl. (In other words, my password was wrong because a portion of it was being recognized as a variable "$xyz", which was blank ;))

*whistles innocently*

The original twitter2lj.pl script is barely recognizable, as I had to redo the feed fetching, parsing, authentication, time(zone) handling, and posting to LJ. However! The HTML format of the output remains (mostly) the same.

Had to rework the feed-fetching-related stuff because the RSS feed has too few items (only 20). The archives XML file, which gives 80 items at a time, is in XML, but not RSS. They require different approaches to authentication as well. I feel a bit guilty about fetching directly from the archives instead of from latest items, but I only do it once a day, so it seems reasonable enough.

Given the problem with the password, the authentication for my RSS feed probably never worked. I just didn't notice it, because authenticating on that feed had no effect, given my settings!

For some reason, @lj_tags = ["tweets"]; was parsed as "array(0xblah)" when posted $lj->Setprop_taglist (in LJ::Simple). However, using @lj_tags = qw(tweets); works. As does using LJ::Simple::QuickPost. It's likely I've misunderstood something about the way lists work in Perl.
ETA: Ciaran has explained it to me :)
( [] indicates an array reference. I got that first line from the original code, which uses QuickPost, which expects an array reference, so it worked for that, but not when I switched to Setprop_taglist. Refreshing my memory about scalars vs lists vs references now *g*)

Now that it works, I have to resist the urge to redo it to optionally use JSON instead of XML (for the sake of those who had trouble installing the XML::Parser, since that would only affect a few people, and of those few people, I'm probably the only one who wants to do Twitter->LJ via this script). I also have to resist the urge to add options to grab other things available via the Twitter API.

So yay, my Twitter stuff archived here: http://afuna.livejournal.com/tag/tweets (backdated so that it doesn't double-spam, forchrissake).

<3 aveleh, murklins, ciaran_h for bearing with my babbling over IM (while I tried to figure out how to best tweak certain things *g*)

I find it interesting that servers respond to the Range header in so many different ways.

My newest toy is a simple Perl script which I wrote to test why certain feeds can't be syndicated on LJ. I've only had a chance to use it twice, but both times the feed couldn't be created because the Range header makes the remote server return 206 (Partial Content) instead of 200 (OK), and each server had a different idea of how it should serve up partial content. And to confuse things even more, some servers return 200(OK) even when the Range header is specified.

First request I tried it with, the remote server returned the feed as a gzipped stream. Second request I tried it with, the server couldn't handle a byte range beyond a certain size (the size of the feed, maybe?); anything larger than that, and it returned 416 (Requested Range Not Satisfiable). I know that the problems with Bad Behaviour are also caused by the range header, but I am not certain about the specifics, other than that it's rejecting the requests because the byte range starts with a 0.

I really should drop this, since there's literally nothing I can do about any of it and there are more productive things I could be doing right now. But. Maybe I'll just check out one more feed if another request about failed feed creation comes in, even if finding out what's wrong isn't likely to help get the feed syndicated.

Enough synning for today! I'm jumping ahead of myself again. I have yet to answer a feed change request where the answer is a simple "yep, valid. Feed changed"; I should get on that soon ;)