Joris' Weblog
Personal weblog with study, work and personal items
2010-08-21
P != NP
P equals NP, that is the question
This month, a mathematician from HP Labs has proposed a proof that might show that P doesn't equal NP. This is a very important question for Computer Science as a whole. It is one of seven the Millennium Prize Problems, for which a bounty of US$1,000,000 is awarded by the Clay Mathematics Institute. Many consider it the most important open question in Computer Science today.
This equation basically tells us whether it is theoretically possible to find any efficient methods to solve some of the trickier problems in Computer Science. Some problems are inherently more complex than others. For example: it is easy to find a particular card in a stack: just go over them one by one till you find it. This problem grows in complexity at the same rate as the stack size. This is considered a P problem. Other problems grow extremely difficult with each added element, like the Travelling Salesman Problem (TSP). In the TSP, the computer has to find the shortest route that visits all specified cities. Because the route depends on the order you visit the cities, and the number of ways you can order the cities grows exponentially every time you add a new city, this problem is nasty. This is called an NP problem.
If the mathematician from HP Labs, Vinay Deolalikar, is right, then there really are two distinct families of problems. There is no hope of ever finding a strategy to quickly find the absolute correct answer to a NP problem that our current computers can execute. The proof is being read by experts in the field and some have come up with some serious critique on Vinay's proof. I am not an expert on this field, so I cannot begin to guess whether he is close to the answer or not.
When the P != NP problem is solved, it would be a great advance in Computer Science, but it wouldn't be the end of it. Its proof could lead to a greater effort to construct new and exotic computers. I don't expect that the resolution of this problem will have a direct influence on my work, but prediction is very difficult, especially about the future (quote sometimes attributed to Niels Bohr).
2010-07-30
Windows 7
Starting all over again
This week, my computer refused to log into Windows XP. The computer seemed to be extremely busy even before any user had logged in. Choosing a user seemed to annoy my Windows and was very sluggish, as if my computer wanted to tell me: “Don't bother me now, I am doing way more important things”.
I tried the solutions that used to work in the past:
- Leave the computer alone for a while, maybe it is some update thing that he hasn't told me about. (I left it alone for a whole day, no change)
- Try to boot into safe mode (with graphics). Same problem.
- Try to boot into safe mode (console mode). Slow, but at least I could do something. I forced a checkdisk, but that didn't make any difference.
At this point I was already fed up with the whole system. I know I could have done at least one other step (using the install CD and try to repair Windows XP), but Windows XP is already 2 systems behind the current version, so I decided to take the step to Windows 7.
I am very lucky because I have a completely weird computer setup. I have three hard disks with an operating system installed on each one of them. I had two Windows XPs and one Ubuntu. Luckily I still had one Windows XP version that I could log into. From there I downloaded a campus version of Windows 7, which was remarkably cheap! Of course, I can only install it on one PC and can only use it while employed at the university, but if everything goes well I will be here for at least another three years, and I suspect that I will be using another system by that time. The download wasn't that intuitive, I got 2 download links without any notice which one to use. I found out by trial and error that one was for Windows 7 32-bit and the other for Windows 7 64-bit. I went for the 64-bit version just in case I ever want to expand my computer memory. The installer that I got was of course also 64-bit, while the old Windows XP was only 32-bit. But after a bit of searching I found that I could burn the installer to DVD and boot from it to do an alternative install.
Well that worked nicely! Except one thing. Windows still doesn't want to know anything about other operating systems, so their bootloader doesn't include my Ubuntu. I searched the internet again, but I am disappointed at how difficult this seems to be. The only easy solution, an unofficial helper program called “Super Grub Disk” couldn't fix my situation automatically.
In the end I used the directions to Reinstall Grub2 from an Ubuntu Live CD.
2010-07-17
If you’re a hammer
Hammering away at software engineering
A while ago at a conference, there was a workshop about software prototyping in Microsoft Excel. I discussed this with a few other programmers and we all thought the idea was really weird. Why would you ever want to prototype software inside Excel?
I was reminded of this workshop when I saw my father write a piece of text in Excel. It made sense, because he wanted to have good control over positioning and he knew how to do it in Excel and would have to figure it all out again in Word.
How about programming? Am I using my own personal ‘Excel’ of programming, because I don’t want to dive into other ways of programming. Is there a perfect way to write software? What are the tools of software engineering?
The days that computers are programmed by changing their connections in hardware are gone. Sure, there are still small devices that are mostly designed and made by soldering the correct components together. Almost all small devices have a programmable circuit however. But even if this was not the case, the programmer can draw the circuit diagram and have someone else solder the components.
The keyboard? No, you can be an expert typist and still not be able to write a single line of code to produce software. I would argue that a true programmer doesn’t even have to know how to type. He could write down the programs on paper and have someone type it out for him.
Surely it has to be a computer language, but how come there are so many languages, and why are they so different from each other? Why does the first use line numbers, the second objects, the third only function calls and the fourth pipelines and blocks? Although a programmer can probably adapt to another language faster than any other person, it is possible that a programmer, when confronted with a new language, will not understand it at all. Programming skill is not about the language, it is more abstract than that.
The programmer’s tools are expressions, combinations and abstractions (according to the famous Structure and Interpretation of Computer Programs). No matter what computer language a programmer uses, he has to have a means to express certain primitive operations, a way to combine multiple primitive operations together and a way to use those combinations as new units.
That’s all nice in theory. It is like saying that woodcarving only needs a sharp instrument, but you won’t find many woodcarvers working with a sewing needle. It is weird to see how much disagreement there is about what makes a language good in the programming world. An old wisdom says that small projects should use dynamic typing and large projects static typing. Of course, Python messes this up by being used successfully in small and large projects.
Maybe software engineering just isn’t mature enough to really know what is best. Maybe the dynamic nature of the craft will never allow us to have the perfect tool. I have worked with C++, Python and Java now and Clojure is on my list of things I should really learn. Hopefully the tool of the future will look something like the tools I have been using up till now.
2010-07-01
Conference on Human Robot Personal Relationships 2010
Personal highlights
Let me start this blog entry by stating that I was involved in the technical support for this conference, so don’t expect an unbiased review of the conference here. On the other hand, I will write about my personal experience of the conference and none of the things I write should be seen as official comment by the organization of the conference.
This year’s conference on Human Robot Person Relationships (HRPR) was held at Leiden University. All the talks were held in the Lorenz Room of the Kamerlingh Onnes building. The conference was held on 23 and 24 June 2010. I really like the Lorenz Room, because of its historic significance and appearance. In this lecture hall (when it was still in possession of the physics department) I followed the lectures of Dr. R.H. Terwiel, who made a great impact on me during my studies.
I really liked the diversity of talks in this conference. There were computer programmers, sociologists, artists, philosophers and theologians present. Let me share a few random thought snippets that I picked up at the conference:
Can we get a better understanding how we look at robots if we study how we talk to them? For example, do we talk about it, with it or with him/her? Is a robot a subject or an object (or a quasi-subject)?
We can see that people are susceptible to virtual relationships if we look at how people deal with online avatars and romantic relationships that form online. Is this a good predictor for how we will feel about robots in the future?
Can we bridge the uncanny valley if artists (and especially theatre artists) help engineers to make their robots more believable?
Do we control the robots, or do robots influence us? How does it make us feel if we are in control or lose control?
How do children think about robots? What should they be able to do? What do they look like?
Can robot personalities migrate to different bodies, so they retain their personality and knowledge of our preferences and yet be able to assist us in very different environments?
It will take a while before the motor control is precise and smooth enough to be acceptable to humans. The first robots won’t be androids, they will most probably be affective objects or maybe pets. Whatever form they might have, trying to construct artefacts like these might help us understand relationships better in general.
For more information see: http://hrpr.liacs.nl/
2010-05-02
First month as a PhD student
My experiences being a PhD student
This April I became an official PhD student at the Leiden Institute of Advanced Computer Science (LIACS) in the Imaging and Bioinformatics group. To get registered needs quite some paperwork and digging in old documents. Prove that I exist (passport), that I have the background (past diplomas), my work experience (CV), educational plan (put together by my supervisor and me). Last week the contract was signed and I am officially a PhD since begin April.
Even though I am working on the same project as I was as a scientific programmer, I wanted to really make a new beginning this April. I changed my focus from the technical details of our current server implementation to the mountain of literature there is about the semantic web in general.
From my viewpoint the semantic web is still very volatile. A lot of things that I considered to be foundations turn out to be debatable or sometimes even inconsistent. For instance: I was shocked to find out that the Dublin Core, one of the more popular ontologies out there, is actually internally inconsistent. Almost a half year ago I visited a workshop about the semantic web in Life Sciences and I was surprised to see that there were no clear standards yet. A lot of people were discussing how to store their data and other practical issues. Of course, that makes sense in a workshop, but after almost a decade you would expect the foundations to be there (or am I just impatient?).
As a PhD, I also got some other nice duties. The least fun is the exam supervision duty, although it is a nice opportunity to read papers with little chance of distraction. A really nice project that LIACS is doing, is setting up a question and answer service for high schools. Computer science is not (yet) a compulsory subject in high schools, and you often see that teachers from other subjects will also teach computer science. LIACS has set up a discussion group were teachers and pupils can ask questions to PhD students about computer science. I really hope this will take off. Lastly I have been asked to review a conference paper. That is a very serious task, and I feel honoured that I was selected as a reviewer. I should note that officially I am a subreviewer, an official reviewer has delegated the task of reviewing this paper, because he deemed that I was more knowledgeable about this particular subject.
I find it amazing that all this happened in the first month of being a PhD. I never expected that things would change that much after working on the project as a scientific programmer. Maybe it is just coincidence that these things happened this month, but I really hope it helps me to get into gear to make the most out of this time.
2010-04-10
Intransitivity
Good – Better – Best
There are a lot of things around us that are well defined ordering. Anything that can be expressed in numbers has a very clear ordering: product X is more expensive than product Y, person A is older than person B, political party C got more votes than political party D. This well defined ordering makes sure that if X is bigger than Y and Y is bigger than Z, that X is bigger than Z. This property is called transitivity. There are a lot of interesting things that are not transitive though.
The school example of intransitivity is rock-scissors-paper. Rock is better than scissors, scissors is better than paper, but paper is better than rock. You cannot assign a numeric value to the items and determine which beats which just by looking which is the bigger number.
Another place that intransitivity pops up is games like Stratego. At first glance there seems to be a very well defined ordering to the pieces. Marshal defeats general defeats colonel defeats major defeats captain defeats lieutenant defeats sergeant defeats miner defeats scout. Then two pieces are added to make the game interesting:
* bombs defeat everything except miners
* everything defeats spies, but spies defeat marshal when attacking.
An example that feels the same, but is hard to express in a very closed form is Hearts. In this card-game you should try to get rid of all your cards with the heart suit, but also lose the queen of spades. At the end of each round, the total number of hearts is added to your score and if you have the queen of spades, you get 13 extra points. The player with the least points wins the game. So far the rules are perfectly transitive, more nasty cards is bad. The intransitive part comes into play when you play by the “shooting the moon” rules. This extra rule says that if you have all the point scoring cards (hearts and the queen of spades) all the other players get 26 points.
The most counter-intuitive example I have found of intransitivity has to be the nontransitive dice. You can make the numbers on six-sided dice in such a way that that dice A has a higher chance to roll higher than dice B, dice B has a higher chance to roll higher than dice C, but dice C has a higher chance to roll higher than dice A. How is that possible? Didn’t I state before that if you can assign numeric values to items it has to be transitive? That is still true. You cannot assign numeric values to the dice as a whole. For a nice of description of nontransitive dice see the Wikipedia page
2009-12-30
Links of 2009
Some of the fun things I found on the net in 2009
Here are some sorted links of things I found in 2009.
Interesting games
http://www.hemispheregames.com/osmos/
http://www.sophiehoulden.com/games/thelinearrpg/
Interesting Videos
http://vimeo.com/3365942
http://vimeo.com/2381662
http://www.youtube.com/watch?v=xdhLQCYQ-nQ
http://www.youtube.com/watch?v=8CrySouTiQk
http://www.youtube.com/watch?v=zFuMpYTyRjw
http://www.youtube.com/watch?v=e7DkeQ0roAM
http://www.youtube.com/watch?v=1ZeciX-3wfs
Comics & Cartoons
http://xkcd.com/602/
http://xkcd.com/681/
http://news.bbc.co.uk/2/hi/technology/8139075.stm
Thoughts & Blogs
http://blog.xkcd.com/2009/09/02/urinal-protocol-vulnerability/
http://blog.tanyakhovanova.com/?p=182
http://strangemaps.wordpress.com/2009/02/26/362-greek-to-me-mapping-mutual-incomprehension/
Most anticipated website
2009-10-25
Win or lose?
Musings on a radio program
Yesterday night, as I was driving home from a party, I was listening to the radio. I listened to a conversation about wining and losing. People could phone in to talk about their personal successes or losses and their opinion on winning and losing. The presenter hinted at the question if it was possible for humans to be able to be unaffected by losses. Why do people need winning and losing? Who determines what a success or a loss is?
One thing really perplexed me. Some people embraced the fact that life is full of successes and losses. Other people phoned in to say that you have to change your perception on successes and losses. That it is wrong to look at life in terms of successes and losses. They claimed that losses should be perceived as successes. It was that last statement that surprised me. If you should stop seeing things as successes and losses, how can you bend a loss towards a success. If you do that, then you are still playing the win-lose game.
What also surprises me, is that I feel uncomfortable every time someone says that they decided to perceive a loss as a success. I know that it is a brilliant coping strategy for them, but it also sounds like an accusation to people who feel miserable. These people suggest that people who feel bad actually lack the willpower or tools to see the good things in their situation. I believe there are bad things that can happen that are just really bad. Some situations just don’t leave much room for optimism. I just hope I don’t meet too many people who claim that there is a good side to everything when I get into such a situation. It is alright to say there are other things that are good, just don’t try to bend everything into a good thing.
I believe these optimistic people are playing a meta-game where your immediate loss is overshadowed by a different triumph: ‘being able to cope with a loss’. The loss doesn’t go away, but you are distracted by the fact that you focus on another battle.
My personal opinion is that the feeling of winning and losing is biologically hardwired into living beings. Living beings who are unaffected by negative things that happen to them, tend to die out. The weird thing is that humans are in some measure able to choose their own battles. They are able to set some of their goals and can sometimes fool their machinery to perceive things in different ways.
2009-09-18
Mathematical Librarian
Inspired by the Dutch Math Girls.
The Dutch Math Girls had an article today about sorting books. I started writing a reaction on their blog, but I noticed that my reaction was getting longer and longer, so I decided to react to their article on my own blog.
I find the process of ordering my books a problem. I would really like to order my books by subject, but there are always books that fall under multiple subjects or elude any attempt to be categorized.
Apart from that, I have many books and little space. This means that I am forced to sort my books by size, to maximize my shelf space. Books of one subject don’t all have the same size to my chagrin.
Ok, maybe I can compromise? Maybe I can just sort my books by size, have neat rows (some of large books, some of small books), but in the row all the books are neatly sorted by size as well. All the other sorting can be done virtually by giving the books numbers and entering them in a database.
To find those books back, you would have to order them by number as well of course, but then we have another problem. How many numbers should I leave between my existing books, so I have enough numbers for future books? This was also a problem in the past, when I programmed in BASIC. Only basic had a nice ‘renum’ function to renumber all your lines to leave 10 numbers between every line. I am not planning to renumber my books!
Another option is to number them with rational numbers instead of natural numbers. But do I really want to look for book 85.12558? If the numbering isn’t approximately linear, then it will slow down retrieval of books.
Ok, so maybe I have to relax my wishes further. No neat ordered rows. I will have rows of small books, and rows of big books, but they are only ordered by registration number. That is the only logical and efficient solution.
But sorting by colour is far more aesthetically pleasing.
2009-08-24
Code Dreams
Dreams to Code
Sometimes, in your dreams, you think you have brilliant ideas. After waking up, the first few minutes, the ideas still seem quite good. A shower later, the ideas might have some nice elements in them, but after breakfast nothing is left of your dream ideas. Most dream ideas are either trivial, or completely impossible.
Last week however, I dreamt about writing some code, which after breakfast didn’t seem too ridiculous. Of course I had to try out the code and I started typing in what I could remember and filling in the blanks with common sense.
The algorithm is terribly slow, but its results are neat. The code from my dream actually creates dream-like images. Here are some examples of the results (click to see larger images):
|
|
|
|
2009-07-26
NDS Programming
Twiddling bits
A while ago I bought a R4DS card. It’s a special cartridge for the Nintendo DS that allows you to use a MicroSD card to use your music, videos or software on your handheld. I have done some other hardware programming on an Arduino and Wiring board and I was interested to see how far I could go on a Nintendo DS.
To start programming I used DevKitPro with libnds. I couldn’t get it working on Windows because I already installed libraries to program my mobile phone, and somehow the two environments didn’t like each other. On a clean Linux system, things went a lot better and I tried following some tutorials that I could find on the web.
Programming the DS reminded me of programming DOS. A lot of the programming effort goes into taming the hardware. Especially when it comes to graphics, you have to plan a lot of things in advance. The screens have to be initialized in a particular mode, which allows you to do some things, and not others. Even with the NDS library, your code will have lines that deal with direct addressing of memory space.
I am surprised at my reluctance to really dive into the matter. I wonder if other people have the same feeling. My favourite programs are very abstract and have to do with the computer hardware as little as possible.
Trail and Error is not really an option on the DS. I decided to prototype a game in Processing first, and after I have everything how I like it, I might give it another go to port it to the DS.
2009-07-19
Random Links
Just some links of this week
To keep up with the technology news, I use Google Reader and subscribe to multiple feeds that I find interesting. This week I have starred multiple items and I want to share them here (the really cool things I usually share using Google Reader).
http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
I still think Python is the coolest language on the block. One disadvantage of Python is that, because it is an interpreted language, it is slower than compiled languages. This usually is no problem with the programming that I do, but when trying game programming or intensive calculations, speed does become an issue. That is why I thought it was handy to mark this project whenever I need more speed for one of my Python projects.
http://www.theonion.com/content/video/obama_axes_pentagon_plan_to_build
The Onion is just funny. Normally I am against spending money on weapons of war, but this time, I think the Pentagon is onto something.
http://www.youtube.com/watch?v=_jZVE5uF24Q
The sequel to Pride and Prejudice and Zombies.
http://blogoscoped.com/archive/2009-07-16-n41.html
I love the Google Doodles. They are small pieces of art and usually the start of wasting another 30 minutes of my time web browsing. It gets worse when I found out there was a hidden signature in the images. Now I have to search everything there is to know about the triforce as well.
http://strangemaps.wordpress.com/2009/07/17/400-japanese-whispers-mapping-the-forbidden-outside-world/
I love maps, especially old maps, when people had to draw them without the aid of GPS or satellite images.
2009-07-12
SFTP and SSH on one server
What to do when you want management access to your machine, but also want users to be able to SFTP to your server.
We decided to go for SFTP. There is much debate whether HTTP(S) is better/faster than (S)FTP. Our reasoning is that most FTP clients deal with time-outs automatically, making it more user-friendly than our previous approach.
Having SFTP open on a server is a security risk. People can put nasty files on your server and SFTP uses port 22 (because it uses SSH) which we also use to manage our system.
In the past we have used a chroot environment with a sshd inside listening to port 22, and our management sshd (outside the chroot) was listening to a non-standard port. I disliked this solution, because I disliked the overhead of two sshd processes and the fact that we had to maintain two operating systems (the regular one and the chroot minimal system). Creating users in the chroot meant changing passwd and shadow files by hand!
Now that we have a new server, I got the chance to start everything from scratch. My first idea was to assign two IP addresses to the server, and hopefully filter ssh traffic by IP. I couldn't make it to work with just sshd configuration however, so I decided to set up my own configuration like this:
Subsystem sftp internal-sftpThe idea is simple. Sshd should act like sftp, and only allow people to sftp in a particular (chroot) directory. Only if someone from my ip with the correct username comes in, the default behavior is overridden and normal ssh is enabled. The last match is for people who try to use the admin account from any other ip. We don't want SFTP access for that user.
ChrootDirectory <my chroot directory>
X11Forwarding no
AllowTcpForwarding no
ForceCommand internal-sftp
Match User <my admin account> Address <my ip>
ChrootDirectory none
X11Forwarding yes
AllowTcpForwarding yes
ForceCommand /bin/bash
Match User <my admin account>
ForceCommand none
A custom adduser.local script makes sure that home directories exist for every ftp user in the chroot directory.
This setup still has to prove itself in practice, so I will be monitoring it closely in the months to come.
2009-05-27
First year work at the university
Comparison between jobs
It has been more than a year ago that I made the job switch from Zest software to Leiden university: time to reflect.
February 2008, I still worked for Zest software as a web programmer. Zest software builds content management systems for middle to large sized companies. In April however a new opportunity opened to work for the university of Leiden as a scientific programmer. It was a very difficult decision to make, because I really liked my work at Zest. After a serious talk with my employers at Zest, I did make the move. Some of the arguments for making the move were:
- being able to spend more time on (personal) research questions
- more theoretical work
- the ability to follow courses at every faculty to broaden my knowledge
But also on a very personal front: the research conducted at the university could indirectly benefit the research on cancer. This is important to me, because my mother was diagnosed with cancer and this allows me to feel less helpless.
Do I really make a difference?
On the large scale, hardly. But on a personal scale, I hope I do. This year, at Leiden university I worked in the BioInformatics group. I was able to attend many meetings of biologists and drug specialists who discussed the latest theories and practices concerning cancer. At the university I am able to access papers that normally require large subscription fees. I have been able to find papers that concerned my mothers case directly (in particular concerning her sensitivity towards carboplatin, one of the better chemotherapy drugs available at this time) and was able to pass them to her specialists.
At the BioInformatics group, we have specialised software to turn image-stacks (like the slices you get out of CT scans) into 3D models. My mother received a CD with her scans on them to exchange the information with another hospital, but I was able to use those scans to turn them into an easier to understand (for non-specialists) 3D models.
To me, those little actions, are important. They allow me to understand and deal with the whole situation.
So am I completely convinced that this was a good move?
That is a hard one. I really learned a lot at Zest, comparable to the things I learned at the university. What I learned at Zest was very practical and useable information. The workflow at Zest was extremely good and I sometimes long back to the fact that everything seemed to work so well together, because everyone had a common goal: doing the projects for our customers. At the university, people are conducting their own research, and while we have a greater common theme, it is still very much one person on one project. I miss pair-programming, stand-up meetings, having a clear todo list and having a big dog around to pet ever so often.
2009-04-22
There be Pirates!
Pirates on the seas and internet
Let us start off by making a distinction between two types of pirate:
- a person who robs or commits illegal violence at sea or on the shores of the sea.
- a person who uses or reproduces the work or invention of another without authorization.
There are obvious differences between the two types of pirates. The sea pirates use violence to get money. Internet pirates make copies of data while they lack the copyright to do so. Also, while the majority of people with an internet connection have at least once in their lives committed internet piracy, a minority of people have committed sea piracy.
There are however also very striking similarities between the two forms of piracy. Both seem to target large corporations, and the pirates seem to think that those corporations have too much power and money. Both types of pirates are difficult to get before a court. For sea piracy the problem lies in the fact that the local authorities do not control the entire region. Again, this has a nice parallel to internet piracy, because no legal body controls the entire internet.
Recent commotions around The Pirate Bay (TPB) have let me to rethink my position on this subject. A Swedish court has found TPB to be guilty on account of the fact that they were an accomplice to a crime. A lot of the defence is actually based on technical obscurity. It is a case of the letter of the law and the pure technical implementation against the spirit of the law and the effects of technology. In the end, I think it was a bad move of TPB to post cease and desist letters and their smug replies to those letters on their website. This showed they were fully aware of the illegal usage and their complete disregard for the effects their technology had. I would still be doubting which side to choose had they taken a far more neutral stance. TPB will be taking this to a higher court, so we will just have to wait what their verdict will be.
Piracy in general seems to try to trigger the underdog emotion. People in small boats or with single computers (but all networked together) take on big ships and large companies. I see it as a form of gradient effect. If there is a large difference of quantity, there will be forces that will seek to bring it back to equilibrium. Just like in nature, structures will also exhibit a form of osmosis, where filters are set up to maintain the differences. These filters will be either extra naval escorts or other ways to enforce copyprotection.
2009-04-04
Tomcat logging
Tomcat is evil by default
Updating to the latest bio-formats seemed to work without problems. When trying it out in our system though, none of our thumbnails were being generated. The weird thing was, that no failures were logged in our log files. Normally images that fail to be processed will show up in our logs so we can see what kind of images we still need to support.
I continued adding logging messages at every step in our functions to find out where the error occurred. Finally I managed to pinpoint the problem to the constructor of an image reader in the bio-formats library.
At that point, I did what I should have done right from the start: I made a small program that didn’t rely on anything but bio-formats. That’s when I found out that there was an extra requirement on an external logger class. I have to conclude that Tomcat swallowed the ClassNotFoundException which I consider one of the biggest sins that a program can do.
Apparently Tomcat has a default setting so that errors go to the console instead of to a log file. If you start up Tomcat with the regular startup.sh, you won’t get the see the errors at all.
Hopefully this tip will help for these kind of problems in the future:
2009-03-21
My goals for Q1
The progress of my goals for the first quarter of this year.
A new year means new promises. Especially when you turn on the television and get bombarded by the positive message of a new American president: "Yes we can!". So, in a "Getting Things Done" fashion, I made some 5 year goals for myself at the start of 2009.
After setting these goals, I cut them up so I have some quarterly goals. As the end of this month draws near, it is time to confess what I did not reach, and be proud about the things I did reach.
These months, I have made more contact with friends I have neglected for too long. This seems as a little step, but when you suddenly realize you haven't contacted someone in months or years, it becomes very embarrassing.
I have also lost weight and am very quickly approaching what I think is a healthy weight. I have switched from regular fizzy drinks to fizzy drinks without added sugar. Every free day, I try to walk for at least 45 minutes (I am in terrible condition).
On the financial side, I didn't reach my goals. I hoped to have saved more, but there were some big expenses that I didn't take into account when I made my goals (most notably a large check-up for my car).
I also wasn't able to finish connecting LDAP to our system at my work. This is going to take at least twice as long as I had thought. I think we need to reassess if it is really worth the time, or if there is a way to make this go faster.
Then there is another goal, which is close to finishing, but is not quite done yet. I wanted to distribute a small game that I programmed myself among my friends. A minimal version is working in my development environment, but it still lacks some important features and I don't know for sure how it will work on normal computers. It is also a bit scary to show the product to my friends. I'm very critical of the results so far.
Maybe I will write about the results of the last goal later.
2009-03-07
Using LDAP in your own software
Usernames don't map easily to Distinguished Names
A while ago I posted that we were going to use LDAP as an authentication service in our project. Now I post about the first pitfall I fell into when trying to implement that in our own project.
In our webservice people authenticate using the traditional username/password combination. Weirdly enough, this got me in trouble right away. When authenticating to LDAP, you need a Distinguished Name (DN), which is something like "cn=Your Name, ou=organisational unit, dc=domain, dc=com". But how do I know which organisational unit someone belongs to BEFORE they log in?
One solution would be to make a very flat LDAP directory structure. But we chose to use LDAP to be ready for growth and having a flat structure doesn't seem handy when you are planning for growth.
The only solution I can think of, is making a special user in LDAP that is allowed to search through LDAP to find the DN of specific user with a particular uid. But I don't understand that such a common use case hasn't been built into the functionality of OpenLDAP.
We are programming with Java, and the library to communicate with LDAP (JNDI) also doesn't offer any help. I keep feeling like I am missing a very important piece of LDAP, but I try to comfort myself in the fact that I can't find any word about it in my LDAP book from O'Reilly or online manuals.
2009-02-18
Top Trumps
From a childhood memory, to a programming excercise
When I was a little boy, I went to England every year on vacation. One year, while staying at a camping, I made friends with a group of English children who used to play a curious card game together: "Top Trumps". They played with a Fantasy themed deck, which as a kid was very exciting and scary. I remember that I drew maps of the world in which the creatures on the cards lived. This could well be the start of my fascination for roleplaying games.
Now, an odd twenty years later, I found one of the Top Trump decks that I bought in that time. This one has the theme: "Goblins & Faeriefolk". I decided that this game could be a nice game programming project. I already made a Tetris game half a year ago, but now I wanted to try game programming again using Pyglet I talked about before.
I have a simplified version of the game working right now, and I am planning to finish it by end March so I can give it to friends to try it out. The game will be single player against the computer only though. I suppose I could make a hotseat two player game, but I don't think this game is exciting enough for my friends to decide to come together to play this. I am just looking for feedback and hopefully it is a way to motivate me to make the game better than it would be if I were just programming it for myself.
Alas, I cannot give it as a download, because it contains the artwork that belongs to Waddington (the company creating Top Trumps at that time). I suppose I could provide the source code without the assets, but I think it would be too much work for people to create their own assets for the game. I will ask my friends if they are willing to experiment and create their own "Top Trump"-style games with it.
2009-02-14
First experiences wih LDAP
My first steps using LDAP
At the university, I am working on the Cyttron Image Database for exchange Project (CIDx). This project is a web-service that allows scientists from the life sciences (mainly Biology and Medical) to share microscopic images, movies and volumes. The difficulty is that these images may come from very different sources (normal microscopes, X-Ray images, computer models, etc.) and the scientists come from different background. It is very hard to see relations between different fields of research, especially if each field uses its own vocabulary of technical terms.
CIDx tries to bridge the gap between different fields and different image types. We are using semantic techniques that people call web 3.0 when applied to the Internet.
Because we are trying to tie things together, our project needs to be able to talk with many different other systems and the past month, the extra requirement has popped up that another system wanted to be able to authenticate users using our database.
We could have written an interface to our database to do this, but because Cyttron might be growing, and because user management isn't really the research goal of Cyttron, we decided to look for existing solutions.
I have to admit, I am disappointed at the existing solutions. The most standard solution I could find was LDAP, and because we strive to use open source solutions, we turned to OpenLDAP. OpenLDAP works ok, but the documentation is very limited. I bought a book by O'Reilly and although it helped a bit, I still think there is a big gap in describing the schema's and their practical usage. Basic information about best practices to fill LDAP with information is sketchy.
Also, OpenLDAP has recently changed the way configuration works, adding to the confusion. I continue to experiment and write a Java API around it, custom tailored to our CSIDx usecase, but it still feels like there is a hole in the market for a system that is a bit more user friendly.