The main reason I left Facebook is because I didn’t feel safe.  Now many of you are thinking that this is related to privacy issues, it is not.  If I was worried about ‘privacy’ in that sense of the word, I wouldn’t be running a blog, under a domain name which directly correlates to my own name, and use it as a pseudo journal.  I signed up for Facebook realizing everything I put into it would be completely public, and never changed my position.  In fact, I originally signed up with a pseudonym Kakistocracy Joe (my usual nick at the time, TBOL3((.)14), or Mr. Pi wouldn’t work), however, I rapidly switched to using my actual name after realizing that I couldn’t keep it a secret anyway, and it just made life harder, so I switched my Facebook name to my actual name, my twitter name to my actual name (well, actually, I just created a new account, oddly enough the old account still gets a bunch of followers), and created this blog, originally on wordpress.com with the domain of leifandersen.wordpress.com.  As such, everything I’ve ever put on Facebook I try to treat as if anyone in the world could see.

The reason I didn’t feel safe was due to their latest feature (at least as of mid August), which is called places.  Now don’t get me wrong, I don’t generally mind services that allow you to public your geographical coordinates, and if that’s all it was, I would have been fine.  However, they also had the feature which allows other people to tag your current location.  I am afraid that someone will be at a place they shouldn’t be, and they’ll tag me as being with them (when I’m not), than in a court of law, that will be used as evidence against me when it shouldn’t have been.  Also, to make matters worse, Facebook doesn’t let you have the ability to ‘untag’ your location, although you can request to have it be removed.  Yes, it is still possible for someone to ‘tag’ me, or for other forms of false evidence to come up incriminating me for something I didn’t do, but this is a start for a more secure environment.

For those of you who say I can just turn that feature off, I know I can.  To be honest, if that was the only issue, than I would have just turned off that feature and continued on with my life.  However, Facebook has a reputation for creating new ‘features’ with very little security in mind.  True, as I said, I generally opt most things to be public, but that doesn’t mean I want a live webcast of everything I do following me 24/7, published on the internet.  Thus, I’m also worried that Facebook will release another feature in the near future that will compromise my security, and not bother to make much noise about it, or I’ll be to busy to notice.  As such, until I get the time to deal with it, I’ve deactivated my account.  Other things such as it being a time sink, as well as it being harmful to other relationships helped in this decision as well.

Also, it wasn’t just a large publicity stunt.  It seems like several large people on the internet such as Jason Calacanis left Facebook only as a publicity stunt, as he never really left Facebook.  Even Leo Laporte who seemed to have been somewhat honest in leaving Facebook came back a few days ago.  My Facebook profile isn’t deleted per say, it’s still in there, it’s only deactivated, and it will remain that way until such a time that I see fit to rejoin it.

Week 8

This week was a slightly less productive week.  I made one or two tweaks to the image diff tests, but otherwise, they’re done.  They can be downloaded here: http://sourceforge.net/projects/blenderrender/files/

Then later today, Daniel (another person from school),  said 10.10.10.10.10, a post I shared on Google Reader, which he apparently reads in Buzz.  Along with my blag, my twitter feed, my google status updates, and who knows what else Google thinks is me (and as such, dumps it in there), as well as the few (non-existent) posts that I actually put in buzz itself.  He also said that the weekly twiiter posts seemed somewhat pointless to him.  (Which is to say, he saw my twitter posts, and then saw my weekly twitter updates which is just a reposting on my blag of what I posted, and thinks it’s somewhat dumb).

These situations made me think of two slightly conflicting things.  First, people actually read what I write.  I find this amazing because hardly anyone besides JT, and many spam bots.  I find this amazing as I have almost put this blag in the list of things no one really wants to read, but will serve simply as a personal ID for me, and I was going to start a new blag, specifically dedicated towards programming/business related things, and keep this blag as a personal archive of my content.  But other people besides me, albeit a small group of other people, also find it interesting.  The second thing that I found interesting is all of the waste we’re putting out on the internet.  I didn’t know my twitter feed was getting posted to buzz (I found out later that it was ping.fm that was posting to buzz, not ‘as’ bad), but then I was also putting it on my blag, which was also going to buzz.  Even more, I never even touched buzz, I was just letting my content trickle into there because I didn’t really care.  So, I’m posting multiple things to multiple services, why?  Because I can?  What I really need, is a single point of entry.  Where I can post both short posts, and long, more thought out posts, like these.  I would use wordpress, but the issues is that if I use a lot of really short posts, it clutters up the whole thing, leaving the longer posts, the ones I really care about, harder to find.  I could use Ping.fm, I could even get it to post to wordpress, but I still have the issue about shorter posts becoming taking over the site.  Also, if Ping.fm goes down, I’m sunk.  Then, what about Reader?  I could share something in Google Reader, and it get’s pushed to Buzz, but nothing else, or I could set it up to post on here, but we’re back to square one about shared items covering up the content I really care about.  So, as I’m making this network, I also have to think about how long these services will last, as many of them have changing APIs, and I would like to have a life outside of keeping these tubes clean.  Also, what if I create a loop of death, say for example, I get Ping.fm to post to wordpress.com, and I get wordpress.com to post to ping.fm (via rss), the internet (or at least a little corner of it, until someone noticed and stopped it) would come to a firey ball of death.

In short, I need a better CMS.  Something that I own, but can ping to everyone else.  Something that can also accept pings of its own (from things like google reader or buzz), and re-ping them, but be smart enough not to post things twice.  Or, is all of this like a soap box, and only a few people are actually reading these narcissistic rants, and should we all get off of our soap boxes?  Although again, I did find out today that a few people do read this blag.

Either way, I’ll let you know what I figure out when I do.  And you can tell me what you think in the comments.

For example, try typing up this blog post on a desktop or laptop, now try typing it up on your ipad…sure, lug around that bluetooth keyboard if you like, but at that point, you have another device to lug around, and it becomes easier to manage a laptop (with the obvious exception of battery life).  Not only will computer scientists, engineers, and hobbyists/gamers be using desktops in the near future, but other trades such as writing, accounting, film/cgi, and many others.

Some have said that computers have become stagnate with little to no innovation.  This is simply untrue.  While the amount of power in a single core has maxed out, and the number of cores in an architecture is slowing, the amount of heat being generated by these computers (and thus by correlation the amount of math they are doing), is still growing exponentially.  Remember people, there’s more to computing that GHz/Cores/Piping/RISC/etc.  Has it gotten less sexy?  Yes, I will admit that, but remember that ‘sexy’ is not a measure of usage, but more a meter of novelty.  Most of the improvements in computers are not in the outside appearance, but in its internals.  Furthermore, ‘sexy’ can also be directly correlated to the amount of marketing a company has put into it’s products, and as we know, Apple Inc. puts in a tremendous amount of marketing.  But to some people, desktop computers are still ‘sexy’.

This doesn’t mean that mobile computing doesn’t have its place.  I recently bought a droid x (on opening day actually), and I love it.  I love the size (although I wish it was a bit bigger), I love the speed, I love it’s power.  However, I don’t use it as a productivity device (yet, I’m working on getting it set up as a tablet in conjunction with my computer), I use it as a portable media player for places like on the bus and the train, which I spend a lot of time on.  I use it for games, but most of the games I play can run on very old hardware anyway, as well as video/audio, and web browsing/email.  I do admittedly use it for some content creation though, for example the picture in this post is a scaled down version of a picture I took with it.  I also enjoy hearing people talking about mobile computing too.

So, in conclusion, while mobile devices are growing, and can be rather interesting, there is still much innovation going on in the desktop/non-cloud-computing arena.  As such, it would be much more enjoyable to listen to podcast/read blogs that talk about that, rather than the annoying amount of ‘cloud’ news that is produced now days.  As such, if anyone wants to show of their websites/products that are not cloud based in the comments, I would be very appreciative of that, and may even give my opinion of it in a future post.

Although Andrea only looked at my code to determine good structure, etc.   I believe that most of the bugs have been removed.  (Although, this is python, and bad code doesn’t show up until someone tries to run it, and I haven’t made regression tests for my regression tests. (at least not yet anyway, and it’s not on my radar in the near future)).

Overall, I think this has been a good week.  I worked almost exclusively on the image comparison diff tester this week.  I wrote it in python, and it can actually work independent of blender (although it does assume that blender is on your system, and in your path, although that can be changed with the –blender-bin flag).  This means that the tests folder can be put in a zip, and placed in the same location as where the current regression tests (for blender 2.4x) are located on the website.
As I said last time, the general way to run the tests is by using python run.py.  You can use the -v flag to see the output of blender as it’s rendering it.  Also, it might be possible to avoid even having to download the images at all, by using the recently built –built-tests flag, which, instead of rendering tests, will render the images and store them in the known good folder.  (It uses the version of blender in your path).
Through the use of the –image flag, you can compare any blender image on your harddrive (currently assuming it’s in the right file structure), this allowed me to make other image comparison tests, that compared blend files built by pyunit based tests.  This (along with the hashcode operator I started putting together last week), means that most of the tools for building good unit tests (and regression tests in general), are almost build (I think).  Meaning that now the only missing piece of the puzzle is the one I was working on several weeks ago (without luck), which is:A.  Getting the python API to work in background mode (which blender currently isn’t designed to do).B.  Getting the python API to work properly in foreground mode (lots of stuff in bpy.wm crash.  (At least it did as of a week ago, do to low bandwidth caps here, I won’t’ merge until tomorrow evening (when I get home), so they may have been fixed by now)).

Although Andrea only looked at my code to determine good structure, etc.   I believe that most of the bugs have been removed.  (Although, this is python, and bad code doesn’t show up until someone tries to run it, and I haven’t made regression tests for my regression tests. (at least not yet anyway, and it’s not on my radar in the near future)).
The comparator is also capable of producing animation tests.  However, this code is slightly buggier.  Also, by default, I’m not running the animation tests in the current regression suite, because it takes the running time up from 2:30, to over an hour.  I have a –animation flag, but it doesn’t work at the moment.  (Although the underlying code works, so it’s basically just a matter of plugging the code into the flag).

Finally, because the animation images make the overall file size ~240 MB (as apposed to 30 MB), they are not in the default build.  But rather, the user has two choices, they can either A:  Render Good ones, or B.  Use a hash code (and download images (or generate them) if something goes wrong).  The hash code also work in images.  However, the code for this is only half built.  The test case classes support it, but the surrounding structures don’t use it yet, this shouldn’t be more than an hours worth of work to implement it fully.

Finally, I didn’t do as much work as I would have liked in the HTML output.  Now, rather than using static CSS scripts from sphynx, I now build the CSS in the script itself.  (With the exception of images, I’m not sure how to do that as a python script…well, there is PIL, but I’d sooner put bamboo up my fingernails than make the needed images that way).  Although the output is starting to look like it’s from blender’s website.  (Although the table doesn’t quite fit).  Also, it will only generate images for single images (well, that, and animations, but it only does the first frame).  Eventually, the animations will be clickable, and you can view a separate HTML page for the entire animation, but that’s not built yet.  It shouldn’t be too hard to make, but I keep having to do one or the other thing, so the current implementation is old, and would probably cause the script to crash if used.

Render Tests

The current set of render tests requires PIL, and python 2.x. In order to run them, inside the blender source directory, look in: tests/render. There, you should find a collection of files. While the absolute path of those files doesn’t matter (aka, you could move the tests/render folder anywhere, and it would do just fine), those files must be kept in the same relationship to each other. If blender is already in your system path, just go into the folder, and run python run.py, and the script will take it’s course. If blender is not in your current path, open up run.py, and change the BLENDER_BIN variable to you blender binary, and than run python run.py. You can view the results in the console, otherwise you can see a list of all of the results, as well as diffs, by opening index.html in your web browser.

(my most recent commits have temporarily broken this as an operator, but I will soon fix that).  There is a webpage that is outputted which allows you to easily see the difference between the renders.  In addition, the algorithm for analyzing the images is much improved.

I also moved my tests.hashcode module into an operator (bpy.ops.tests.hashcode()), or Tests->Hashcode.  Although it still only gives feedback in the terminal, and only takes some things into account when making the hashcode.

I’ll try to make this short, as it’s late here.
Even though I got sick this week (it turned out to be just a cold/flu), and went out of town (and still am), I believe that this week has been one of the most productive weeks so far.
This week started off with me implementing some of the GUI features I worked with last week, which is to say I started making the image diff tests into an operator.  (Previously they had been baked into the tests directly).  They could be run from a Tests dropdown menu, although there was non feedback outside of the console, leading the user to believe blender had frozen while the tests were running.  Andrea pointed out that this was pointless, and that it should just be a script.  As such, I started implementing it in all in python 2.x.  The tests can be found in tests/render, and I put together a slight howto on how to run them: http://wiki.blender.org/index.php/User:LeifAndersen/GSoC2010/Howto
Render TestsThe current set of render tests requires PIL, and python 2.x. In order to run them, inside the blender source directory, look in: tests/render. There, you should find a collection of files. While the absolute path of those files doesn’t matter (aka, you could move the tests/render folder anywhere, and it would do just fine), those files must be kept in the same relationship to each other. If blender is already in your system path, just go into the folder, and run python run.py, and the script will take it’s course. If blender is not in your current path, open up run.py, and change the BLENDER_BIN variable to you blender binary, and than run python run.py. You can view the results in the console, otherwise you can see a list of all of the results, as well as diffs, by opening index.html in your web browser.
(my most recent commits have temporarily broken this as an operator, but I will soon fix that).  There is a webpage that is outputted which allows you to easily see the difference between the renders.  In addition, the algorithm for analyzing the images is much improved.
I also moved my tests.hashcode module into an operator (bpy.ops.tests.hashcode()), or Tests->Hashcode.  Although it still only gives feedback in the terminal, and only takes some things into account when making the hashcode.
As far as next week goes, I am going to spend a bit of time with the webpage.  It’s not quite flush with blender’s webpage, but it’s getting closer.  But more importantly, I can see a few more options that would allow the user to get more of the raw data behind the images (which I would assume would make it debug).  Most of this I believe can be written in HTML, but some javascript may be useful.  I also plan to get animations working with these tests, and possibly bringing this back in to work with the blender operator.  I also am going to try to get a lot more feedback this week, and hope to have the tool used soon after.  I will also spend a bit of time getting the hashcode operator to take more into account, as well as giving more feedback to the user.  (As in feedback that’s not only just console output).  I would also work on aggregating the data in the blend file for the user to test, but blender is already capable of that.  Finally, I plan on improving the integration of Ctest and CDash, but that seems trivial to do.

This week has been one of my better weeks.  As usual, there were a few bumps this week, but not too many.
This week dealt with two things:1.  Image comparison2.  UI, and thus wrapping unit tests in operators to insert into the UI.
Per Matt Estela’s recommendation, I tried incorporating Perceptual Image Diff http://pdiff.sourceforge.net/ into the tests to compare images.  The pre-compiled binaries work fine, however, compiling from source presents a bit of a problem, even with all of the listed dependencies installed, it complains that several printf stamens are not declared.  I emailed the developer and am waiting for a response.  This isn’t my main priority.  However, I will keep using PIL until such time as a tool like this works.
Getting tests in blender’s UI represents some challenges that do not exist when using CTest as a testing platform.  First of all, I thought a lot about render tests.  If we use blender’s current set of regression tests that blender has, it shouldn’t be very difficult to create a tool that allows a developer to render a blend file, and than compare it to another image file, and than write another tool to run all of the tests.  The problem is that putting this into the GUI, which requires python 3.  And on top of that, developing a good UI for this sort of thing can also be a bit tricky.  Fortunately though, it still is trivial to wrap up unit tests inside an operator, so ultimately, running the tests won’t be too big of a deal, as apposed to setting them up.  I also toyed with writing the operators required in C, although this isn’t as good of an idea as writing it in python, where we could simply give developers a template file to use for their tests, but it may still prove useful.
Next week should be interesting.  Starting on Wed.  and going to the end of the week after that, I’ll be out of town.  With that being said, I won’t be out of communication, but rather, I’ll probably be spending less time on IRC.  (I’ll still try to attend the usual Sunday meeting, as well as spending an hour or so online though).  I will respond to emails as usual, albeit probably a bit slower.   I’ll post an email specifically about this to this list in case this portion of the review is missed.  I still have more UI stuff planned next week, but mainly merging the UI and rendering algorithms into one fluid tool, at which point I’ll start automating it.

So, just a brief checklist of what has happened:

1. Project Started
2. Gtest set up
3. Pyunit setup
4. tests module created
5. Hashtests and image comparison tests built
6. Learned a lot about how the Blender codebase works
7. Made some hacks and bug reports with the Blender codebase
8. Initial work on GUI panel for tests (originally dropped, but recently brought back into the foreground)
9. CMake now using CTest and CDash to report the project

There was probably some more, but those were the biggest.  When I look back on that, my initial impression is that I really could have gotten more done.  In particular, most of this stuff is samples, I have been working on the bones behind this framework, and the metaphoric meat has yet to come, although it did start to at the end of the month.  However, it than becomes important to look back at the schedule planned for this project.

At the time of writing this, the predicted portion of work is supposed to be:

June 8 – June 23

This time should be dedicated to development of other end to end and regression tests. It is much longer than the first suite because much less time was spent on it during the community bonding period. An emphasis will be put on making various forms of render based tests. At this time, it will be important to determine a good strategy for how often the test suite should be run, and, if needed to be run in multiple increments, i.e. some tests running with every commit, while others run only nightly. This section can be further split into two sections. The first being setting up the testing platforms, and the second part being creating a sample test suite for the built platforms. Note that the task of scouting for viable platforms, based upon what Blender currently uses, and the toolset others, should be a ‘back burner’ project, while the other sections are taking place, and should be completed by the time that this section starts.

So, as far as the project goes, it looks like I’m on schedule, with one exception, I have very few tests in hand, do to the amount of time I’ve spent trying out testing frameworks.  I don’t think this is a very bad thing per say, not only was that particular timeline fairly preliminary, but it also gave a lot of time for testing frameworks.  So, in order for me to stay up to date, this next month will be spending a lot more time developing tests, and a lot less mucking about with how they are presented, especially as most of the frameworks have already been set up, with two exceptions, registering it as GUI within blender, and allowing for easy automation of new tests (currently a developer has to manually plug their tests into the system, at which point they will be automated).  Also, it might be a good idea to make the pyunit tests operators in the near future, which should allow for them to be easily made into a gui.  With the addition of the current regression tests that Blender has, this could be vary useful.

As far as participating in the community, I believe that that is going well.  I’m idling in the channel, and have had several conversations about problems with the code.  At least, I think I’ve done a good job, this is one of the situations where unless someone is telling you that you need to do better, it’s hard to know that you can.  I’ve also participated via email and the bug tracker.  I don’t see a need to make many changes for next month.  With that being said, for about a week and a half, possibly two and a half weeks next month, I’ll be out of town.  I’ll still be working on my project, and responding to email, but do to poor internet access, I’ll likely not be doing too much idling on #blendercoders.  Although I’ll try to be on for an hour or two a day in the evening (CEST time, morning for me in MDT).

So, in short, I believe I’m still in the window that makes me on track.  I’m not at the front of the window per say, but it’s not anything I think I need to worry about, yet.  Also, I’ve been speeding up, so hopefully I’ll get ahead of my proposed schedule soon.  I will be spending more time developing tests, and than plugging in blender’s current regression tests (probably by using the same file structure those tests are currently in), and I’ll be keeping my current pace in the community.

I spent a lot of time this week trying to get some of the tests to work in background mode. Unfortunately, several of the needed api calls (saving/loading/rendering/saving images), only work when in regular mode. I also had other problems with API calls not working properly even in the foreground mode, and I’ve done some more hacks on the codebase, and submitted some bug reports, although at this point, I’ve got most of the API calls I need working, albeit they’re still a bit buggy. In the end, we’ve decided to run the tests in the forground (for now), and just run the exit blender api call at the end of the test. The largest problem with this is that if the test segfaults, blender won’t quit, and the user must quit it himself. Also, it’s quite annoying to have a window pop up and close over and over again while running the tests. Thus, as soon as some of the bugs I’ve reported are fixed, I’ll be switching back to background mode.

I’ve also started doing some image comparison tests. I don’t really have any good algorithms yet, so I’m still very open to suggestions. Currently though, I’ve tried several iterations of a histogram comparison. Both comparing all the color bands at the same time, and comparing them separately, the first one is quite a bit faster, although I think that’s do to how I’m using the API calls.

By far, the largest problem I’ve run into image testing (besides there being no python API call to render an image and save it, even in foreground mode), is the fact that PIL currently only supports python 2.x. So, to compensate for this, I’ve split the image tests into two parts, the rendering test (and everything else that can be done in blender), and the comparing tests, which use PIL. If I need to, I suppose I could read in the files myself, and use the raw data the files bring in, but I believe that would waste time, and wouldn’t be as effective as using something like PIL. The issue than becomes the lack of not being able to render anything and save it in python. To get around this, I’m currently saving to a .blend file, using the command line to render the files, although I’ve run into the problem of getting CTest to run multiple commands in one test, which I’m looking into. Another thing I’m thinking about doing, is getting an exec function for the image.save_as() operator to work. Oddly enough, it only works when the image screen is up (which, in my opinion, goes against blender’s non-modal philosophy), or rather, when the image is in the buffer, (which only happens after the user hits f11 or f12 (+ ctrl, or whatever you set to render/show images/animations), and the problem comes because there is no python API call for it. Oddly enough, in regular mode, the invoke function also works just fine as an exec function, but in background mode, it segfaults. I’ve stopped it from segfaulting, but because the value is still null (do to being in the background), it’s still imposable to actually save the image. (Although the ‘view’ is the image view). Also, the invoke function still is a bit quirky as an exec function. So, I’m thinking it might be possible (because I’m currently running the tests in the foreground anyway), to just render the images and save them in the python file, which would not only be easier (api call wise anyway), than mucking about with CMake, but make more logical sense, and allow for easy unification of the tests when PIL is supported for python 3.x.

Most of this I’ve done to test what appears to be rendering problems with viewing hair as ‘object’ or ‘group’. (Which may have been solved, or proven to be a feature rather than a bug, I haven’t checked the bug tracker for that particular bug for several days). But, as long as it’s not a feature, it still would be useful for a regression test.

I did get a chance to look at sphinx documentation, although not very much, and not enough to get anything done. I doubt I’ll get to it next week, but I still am keeping this as a back burner project. (Albeit a very back burner project, it’s currently more important to me to get the python api up to snuff in background mode).

Finally, today and/or tomorrow, I plan on writing a month in review. Because I like wordpress’s editor, I’ll probably first put it up on my blog (http://leifandersen.net), and shortly post it on my project’s wiki page: (http://wiki.blender.org/index.php/User:LeifAndersen/GSoC2010)

Next week I do not plan on dealing to much with issue of background mode, except as a back burner project. I have submitted bug reports, and will be participating with them though. I may devote a few hours next week to this though. I also plan on finishing up good image comparison algorithms, and I’ll put them in my tests module, however, I won’t be able to use that module until PIL works on python 3.1, but it should make it easier to move the tests. Except for possible rendering problems, I still believe that hashcode based tests are more effective than render tests, so I’ll probably work on that a bit. Finally, I plan on making the structure cleaner. I may remove the gtest portion altogether, as it’s not needed, and just in the way, as well as possibly changing the structure of the python tests to a more logical order, rather than simply module level order. As well as finding a good place to put the CTest commands.

Now, while the Nintendo 3DS doesn’t require glasses, it still expects the user to have two eyes.  Because I have but the one eye at my disposal, the Nintendo 3DS is not usable to me.

While I am not as outraged as the person in this video, it does encapsulate the spirit of how I feel.