Notes on a kind of string equivalence

The problem I had been batteling with started when a friend was asking for me to write a program that could match the headers of differnt columns in some CSV file. They would be really similar but not so similar that casting both files headers to uppercase would resolve it. An example might be “CurveConfig1”, “Curve Config 1”, “curve_config1” and “curveconfig1”. My first ideas was that I could convert these down to the last format in that list. You would have to convert both lists of headers to the finnaly format as you can't go back the other way. Once they are all in this 'normalised' format they can be compared directly.

The following is from my notes at the time:

Question 

Does encoding down to the lower information state make the comparison less reliable and should there be a hierarchical, not quite probability level, value stating the confidence of the comparison?

If we did then: 

 “CurveConfig1” -> “CurveConfig1” would be 

 “Curve Config 1” -> “CurveConfig1” would be 

 “CurveConfig1” -> “curveconfig1”

I recently came back to this after not having looked at the problem in over a year and thought it would be a good experiment to see how quickly I could code up something this simple. I was surprised that it didn't take too long. It's the perfect experiment in which to use TDD and I'm glad I did because I think that was core to how I sketched it out so quickly.

The following is what I came up with in python:

The functional code being:


I then thought since this is a nice concise function that it would be interesting to see how long it would take me to sketch this up in a second language. I do find it frustrating when simple functions like this are spoken about online without something embedded to play with. It didn't take too long to convert but the conversion really points out the scale of built-in functions to python.

Javascript version:

Comments

Post a Comment

Popular posts from this blog

Structural engineering with cardboard

Image
Moving into a new flat I have ended up with a lot of cardboard and have actually struggled to get it to all be taken away. I then saw this Kickstarter on hacker news which introduced me to the idea that I could perhaps use the cardboard I had been throwing away the whole time. I have been toying with some ideas for building myself a robot, or at least a robot arm but they seem to be just ludicrously expensive and still not able to lift much weight. One of the problems with the advancement of robotics is that although processing power has come down in price at a rate that no other product ever has, the cost of actuators (The motion inducing components like motors) has remained largely the same. I want to explore an idea I have for using cheaper motors in a robotic arm in another post though. The problem I want to look at here is the possibility of being able to use cardboard, a material which can almost be considered to have a negative price, as the structure to the robot. I love the ...
Read more

The twelve fold way

Image
I really struggle to remember the combinatorial rules. Do I need to use choose or do I need to use permute? How do I calculate each of them? Is this a case of stars and bars or is this something else? One of the things I want to add to my chatbot program is a way to ask which of the many systems should I be using? I did once try to build a javascript tool although I don’t think it ever really made itself into a useful state. I recently found a paper called “The twelvefold way” which explains all the different ways you put balls in boxes (or urns as is used in the paper). It really helped me to understand, at least a little better, how to calculate the different permutations of the problem. With this newfound understanding, I once again had a go at producing a program to calculate each, with a simple form-based input. One of the things that I wanted to extend this to doing was informing me of what the calculation method was. I really like the way google now lends a hand when you a...
Read more