Any good surname databases?

Question

I'm looking to generate some database test data, specifically table columns containing people's names. In order to get a good indication of how well indexing works with regard to name based searches I want to get as close as possible to real world names and their true frequency distribution, e.g. lots of different names with frequencies distributed over some power law distribution.

Ideally I'm looking for a freely available data file with names followed by a single frequency value (or equivalently a probability) per name.

Anglo-saxon based names would be fine, although names from other cultures would be useful also.

@PengOne I don't think you can download their dataset, though, and it's name origin not frequency. — Rup, Jun 13 '11 at 15:04
@Rup: I didn't look at it much, hence why it's a comment and not an answer. I figured it might be a place to start looking. — PengOne, Jun 13 '11 at 15:05

redcalx · Accepted Answer · 2016-02-05T13:13:38.903

I found some US census data which fits the requirement. The only caveat is that it lists only names that occur at least 100 times...

Found via this blog entry that also shows the power law distribution curve

Power law curve in surnames(blog entry)

Further to this you can sample from the list using Roulette Wheel Selection, e.g. (not tested)

struct NameEntry
{
    public string _name;
    public int _frequency;
}

int _frequencyTotal; // Precalculate this.


public string SampleName(NameEntry[] nameEntryArr, Random rng)
{
    // Throw the roulette ball.
    int throwValue = rng.NextDouble() * frequencyTotal;
    int accumulator = 0.0;

    for(int i=0; i<nameEntryArr.Length; i++)
    {
        accumulator += nameEntryArr[i]._frequency;
        if(throwValue <= accumulator) {
            return nameEntryArr[i]._name;
        }
    }

    // If we get here then we have an array of zero fequencies.
    throw new ApplicationException("Invalid operation. No non-zero frequencies to select.");
}

Although in hindsight it's more efficient to use a binary search approach on an accumulating list of frequencies. Or for fixed numbers of samples you can use stochastic universal sampling (http://en.wikipedia.org/wiki/Stochastic_universal_sampling) — redcalx, Jul 01 '11 at 21:32
The links in this answer are no longer valid. 1990 name data is available: http://www.census.gov/topics/population/genealogy/data/1990_census/1990_census_namefiles.html — Doug Lampe, May 12 '15 at 18:20

score 4 · Answer 2 · answered Jun 13 '11 at 15:28

4

Oxford University provides word lists on their public FTP site as compressed .gz files at ftp://ftp.ox.ac.uk/pub/wordlists/names/.

answered Jun 13 '11 at 15:28

Russell

141
3

Thanks. There's no frequency data there but I could sample from a given list with an appropriate distribution to give more realistic test data. – redcalx Jun 13 '11 at 15:51

score 3 · Answer 3 · edited Feb 10 '15 at 20:01

3

You can also check out jFairy project. It's written in Java and produces fake data (like for example names). http://codearte.github.io/jfairy/

Fairy fairy = Fairy.create(); 
Person person = fairy.person();
System.out.println(person.firstName());           // Chloe
System.out.println(person.lastName());            // Barker
System.out.println(person.fullName());            // Chloe Barker

edited Feb 10 '15 at 20:01

MariuszS

30,646
12
114
155

answered Nov 22 '13 at 11:48

Jakub Kubrynski

13,724
6
60
85

Any good surname databases?

3 Answers3