Re: Generating Unique Numbers - C# Discussion Boards

25-Jul-07 2:55

Hi,

I would like to make an application that generates unique numbers (numerics only) in sequential order (better if this way but not compulsory).

It should be able to recover in case of a crash. i.e. if the generation crashes or system crashes, then the next time its up, it should give unique numbers only, and not repeat any number previously given. This is of utmost importance.

And if crash recovery feature doesn't involve I/O, it would be better. (e.g saving numbers to disk and stuff).

Note: Random is not mandatory, unique is.

If you have a solution to this, please help.

Thank you.

Colin Angus Mackay25-Jul-07 3:01

25-Jul-07 3:01

Use GUIDs and convert them to numeric - it is a sequence of HEX anyway so it can be converted quite easily.

Upcoming events:
* Glasgow: Mock Objects, SQL Server CLR Integration, Reporting Services, db4o, Dependency Injection with Spring ...

"I wouldn't say boo to a goose. I'm not a coward, I just realise that it would be largely pointless."

My website

Luc Pattyn25-Jul-07 3:48

Luc Pattyn

25-Jul-07 3:48

Hi Colin,

this is puzzling me: I expect GUID are unique all over the universe.
They also need many bits; how can it be if a take 1 million GUIDs and convert each of them
to int, that all ints will also be unique (inside my confined process that is),
so which transformation preserves uniqueness while compressing from many to 32 bits ??

A simple hash code wont be sufficient, its only function as I understand is putting objects
into 2^32 buckets with a "good distribution".

Can you enlighten me on this ? Thanks.

Luc Pattyn

try { [Search CP Articles] [Search CP Forums] [Forum Guidelines] [My Articles] }
catch { [Google] }

Colin Angus Mackay25-Jul-07 4:03

25-Jul-07 4:03

Guids are 128 bits. No one said anything about being constrained to 32 bits.

Luc Pattyn25-Jul-07 4:11

Luc Pattyn

25-Jul-07 4:11

Thanks Colin

I confused with the GUID strings, the ones used for registering a class (and filling the
registry).

Now 128-bit isnt very much, and MSDN on Guid.NewGuid says "There is a very low probability that the value of the new Guid is all zeroes or equal to any other Guid." which may not be good
enough for the OP, he really insisted on uniqueness...

Smile | :)

Luc Pattyn

try { [Search CP Articles] [Search CP Forums] [Forum Guidelines] [My Articles] }
catch { [Google] }

CPallini25-Jul-07 4:19

CPallini

25-Jul-07 4:19

Oh, the OP don't have to worry about it...
However, using GUIDs is not practical due to timing requiremts and 128-bit integers handling: he don't need such complexity.
Smile | :)

If the Lord God Almighty had consulted me before embarking upon the Creation, I would have recommended something simpler.
-- Alfonso the Wise, 13th Century King of Castile.

CPallini25-Jul-07 4:15

CPallini

25-Jul-07 4:15

Your solution is simply not practical.
See Luc Pattyn one [^] for a better approach (at least IMHO).
Smile | :)

BTW you don't need to convert hexadecimal chars to obtain integers from GUIDs.
Smile | :)

If the Lord God Almighty had consulted me before embarking upon the Creation, I would have recommended something simpler.
-- Alfonso the Wise, 13th Century King of Castile.

Colin Angus Mackay25-Jul-07 10:43

25-Jul-07 10:43

CPallini wrote:
Your solution is simply not practical.

I realise this also[^]

Jalpesh B. Patel25-Jul-07 4:01

25-Jul-07 4:01

Converting GUID into numbers does not guarantee uniqueness... I've checked that already.

Colin Angus Mackay25-Jul-07 4:05

25-Jul-07 4:05

Jalpesh B. Patel wrote:
Converting GUID into numbers does not guarantee uniqueness... I've checked that already.

Yes, it does. You have to translate it in to a 128bit number. You never specified you were constrained by a specific size of number. If that is the case you might want to be a bit more specific about your requirements in the future.

Jalpesh B. Patel25-Jul-07 4:08

25-Jul-07 4:08

I am sorry for that...

but mine should be in the range of Long or uLong.

Colin Angus Mackay25-Jul-07 4:10

25-Jul-07 4:10

You might want to go with Luc's idea[^]

Jalpesh B. Patel25-Jul-07 4:10

25-Jul-07 4:10

By the way,

How do i convert it to 128 bits ?

Justin Perez25-Jul-07 3:02

Justin Perez

25-Jul-07 3:02

This sounds like a homework assignment. If there is no In/Output, how do you expect to ensure that these numbers are unique?

After a crash, that is.

I get all the news I need from the weather report - Paul Simon (from "The Only Living Boy in New York")

Jalpesh B. Patel25-Jul-07 4:05

25-Jul-07 4:05

Its not homework man.

I really have to make an app that can server as much as 50,000 numbers per sec.

I/O makes it too slow...thats why

Dan Neely25-Jul-07 4:43

Dan Neely

25-Jul-07 4:43

You'll need something longer than a 32bit ID then.

50,000ids/sec = 4,320,000,000ids/day, so would overflow a 32bit ID in slightly less than 24 hours.

--
You have to explain to them [VB coders] what you mean by "typed". their first response is likely to be something like, "Of course my code is typed. Do you think i magically project it onto the screen with the power of my mind?" --- John Simmons / outlaw programmer

Chintan.Desai25-Jul-07 3:07

Chintan.Desai

25-Jul-07 3:07

store number 0 to text file before execution of ur application.During execution get that number from the file.Increment that nos. by 1.and overrite it to same file. Rose | [Rose]

Regards
Chintan
www.visharadsoft.com
(Nothing is so purify as KNOWLEDGE)

Jalpesh B. Patel25-Jul-07 4:03

25-Jul-07 4:03

Tried it already...

This is too much I/O at this rate I would generate between 30-50 per sec.

I want 50,000 per sec.

martin_hughes25-Jul-07 3:10

martin_hughes

25-Jul-07 3:10

It's very difficult to maintain state across application crashes without resorting to some form of external mechanism in which to persist that state (IO/Database/WebService/Whatever). However, have you considered the use of GUIDS in your application?

"It was the day before today.... I remember it like it was yesterday."

-Moleman

Jalpesh B. Patel25-Jul-07 3:57

25-Jul-07 3:57

The problem with saving something to disk or database is the reduction in performance of the application.

I have to generate the numbers as fast as 50,000 per second. Without loss.

I have tried saving in registry, database and files, the i/o overhead is too much. I didn't forget MSMQ too.

And converting GUID to Long does not guarantee uniqueness.

Please help with something else.

martin_hughes25-Jul-07 4:17

martin_hughes

25-Jul-07 4:17

I'm intrigued - what does your application do that it requires 50,000 unique numbers every second?

"It was the day before today.... I remember it like it was yesterday."

-Moleman

Jalpesh B. Patel25-Jul-07 4:27

25-Jul-07 4:27

Well, actually other apps request mine to give them a unique number. and those request come very very fast...

martin_hughes25-Jul-07 5:20

martin_hughes

25-Jul-07 5:20

Ah, I see.

Here's an idea (but I think Luc's is better, now that I've just read it). The major problem you're facing is the IO overhead, and yet you do need some way to prevent duplicates from occuring.

How about booking out blocks of numbers in large chunks (for the sake of argument, say 50,000 at a time), and keeping a file which tells your program the range of the next chunk. That way, if your program crashed it could restart and know what the next chunk of numbers should be, and the number of IO requests would be fewer than with an update after every number issued. Of course, in a situation where your program crashed after only 10 numbers had been issued, you'd have "lost" 49,990 numbers...

"It was the day before today.... I remember it like it was yesterday."

-Moleman