|
Such fix already was posted. It not solution. Cause we lose about 90% of speed.
|
|
|
|
|
With my initial protoco design, I have been passing data via the primitive data field types (strings, int's, etc). The basic logic flow requires me to:
1) Deconstruct the object and assemble the varying field values into a CNDKMessage.
2) Send that message to the server and reconstruct the object on the server side.
3) Do some processing.
4) Deconstruct the object again assembling its different values into a CNDMessage and send it back to the client.
5) The client then takes the message fields and recontructs the object with the new values.
<sigh>
I have been looking at the CNDKMessage that will allow me to pass an entire send an entire object (LPVOID,UINT size).
This would make it much more convenient to just send the entire object without having to compose/decompose it's primitive elements.
I don't see any troubles with this. But am I missing some design considerations here? The only one that I can think of is that this may cause troubles when producing a cross-platform client.
How does everyone else generally do it? Do you usually just send the entire object, or do you decompose your protocol into primitive types of integers and strings?
Thanks in advance.
Jim
QTExtender - The OFFICIAL addon for QuoteTracker.
|
|
|
|
|
I decompose the message into primitive types.
Sébastien
|
|
|
|
|
Hi Sébastien,
Is there a way to get the OnIsConnectionAccepted() function to behave in the same fashion as if the server were not running?
If I return FALSE, the clients still appear to connect but they cannot send or receive messages and in some cases, using the GetNbUsers() function to limit the number of users has caused my server to crash.
Any ideas?
Thanks,
Don
|
|
|
|
|
Hi,
Please remove the line ASSERT(GetNbUsers()) in the method OnIsConnectionAccepted.
It should work now.
Sébastien
|
|
|
|
|
Hi Sebastien,
Thanks for getting back to me. Unfortunately, I'd already tried that. All I use doing in the OnIsConnectionAccepted method was the following:
if (GetNbUsers() > LicenseCount)
return FALSE;
else
return TRUE;
Fortunately, I found a better way to accomplish the license limit. I always return TRUE in the above method and go ahead and let them connect, then I test the number of users in the OnMessage method when they've joined. If they've exceeded the license limit, the server sends them back a license error message and the client informs the user, does a CloseConnection(), an OnTimer event is started, and it will attempt to reconnect in x milliseconds (defined in a registry setting).
This gives me a little more control over what's displayed to the user on the client side, and the administrator on the server side, where I can use the AddSystemText method to display what just happened. It shows that the user attempted to connect, but that only x number of users are able to connected with this particular license, then lets the admin know that it'll be disconnected, and then you see the user leaving the server. It all happens within a fraction of a second of course, but it's perfect for what I was trying to accomplish.
I'm not sure why the OnIsConnectionAccepted() method wasn't working. Probably something I've done. Anyway, thanks again for this great class!
Cheers,
Don
|
|
|
|
|
hi everybody! (hi doctor nick! )
ok, herez the thing.. i'm working in MFC for one year, but i've never worked with sockets yet (except with VB, which is kinda gaay..JK ).
i'm planning to make an instant messenger on this concept:
app consisting of server and clients. server accepts new connections, establishes a new connection with the requesting socket and voilá - chat. also, when my users want to connect to somebody's server, program creates a new socket and voilá - chat. so, herez how i immagined it, but i seem to have no ideas how to implement this , and now i'm hoping someone here will help me with those ideas..
user is able to exchange messages with many users at the same time, but they talk separately.(like...icq, msn, yahoo messenger..). each connection should have its own dialog, with separate properties. there is a main dialog which would basically organize everything (accepting connections, taking care of the existing ones, cleaning up when shutting down, holding contacts..).
and that's just basic stuff. how do i search the net for online users?? if a user wants to search for his frend, how do you that? and do i search by machine name or...HOW...!?!?
I'd just like to say that i'm not asking for the code (not jet )), just for the concepts how this is done. i've never done anything like this, and i'm clueless..
oh and, thanx everybody, this site has a GREAT community..
t1
---
kick ash.
http://t1tan.cjb.net
|
|
|
|
|
Hi t1,
I was in your shoes about six months ago, still don't know dookie about client/server winsocks and such, but this class will definately send you in the right direction, if nothing else.
My advice is to take a little time and play with the demo. Everything is documented pretty well and you should be able to do all of the things that you mentioned here.
I believe someone had asked the question in one of these previous posts on how to get the list of users to a client, and the answer was to put them into an array and send it back to the client from the server. You'll have to see the post for more information on that question in particular, but basically, your client would send a message to the server containing the users name that you're seeking (or request the complete list if you wish), and then the server could look up the name and return a TRUE or FALSE (or whatever you want) letting you know whether the user is connected or not.
You could also set up new messages to be shared between the client and server, in which the client would tell the server which userid to send the message to, and then the server would simply take the message apart, see who it's supposed to go to, put it back together, and send it to that user, and vice versa. If the user isn't connected, the server could send a message back to the requestor letting them know.
One thing to note (something that got me) is that the messages you define have to be in order and identical on both sides in ChatMessage.h (in your client and server). I was banging my head against the wall for a few days with this one, until it dawned on me.
Hope this helps, or at least encourages you to press on. It's a little intimidating at first but if you take a little time and familiarize yourself with the NDK functions above, you'll be chatting and winsocking in no time!
Cheers,
Don
|
|
|
|
|
everything's kewl, except for one thing: i was planning to make a peer-to-peer app, so there should be no central server at all. that's when i get into trouble of getting the "online users list" - where to get it from?? how do the programs like kazaa & winmx do it? (as far as i know they don't have any server??) and how do you manage everything (dis)connections if there is no server?? and how to connect two "clients" (i.e. no real server)?
thanx for the rest of the tips, i'm sure they will prove usefull
there is one more thing i could ask here.. if i create a dialog resource, and inherit it from CNDKServer/CNDKClient, is that more practical then creating a public varible of the same dialog class??
that's it, i've already said too much ...
tnx
t1.
---
kick ash.
http://t1tan.cjb.net
|
|
|
|
|
Well, I'm mostly guessing and basing a little on experience, as I haven't seen the Kazaa code, but I imagine that the applications are acting as both clients and servers.
Peer-to-peer usually occurs by specifying a specific computer to communicate with, in which case you would need to know the IP address or the hostname of the computer you're calling. When you make the request, it'll send a packet to that computer to see if it's listening. If so, the computer sends back an acknowledgement and the connection is established. This all happens on a port that is specific to the application.
It looks like Kazaa and these other chat-type programs operate a little differently, where they don't know the IP address or hostname of the computer they're calling, so they more or less broadcast the request by enumerating IP addresses and attempt to locate the applications that way.
In Kazaa's case for example, and again, just by what little I've read on their website about Supernodes and such, it sounds like when you install the application, it detects if you have a high speed Internet connection. If so, the "server" part of your application is probably enabled and your application will listen on a predefined winsock port, as well as send connection requests, and will store a list of connected users and local files that it finds.
As others start up in your area, they'll do the same thing and ask you for a list of your users, and so on.
When you start the application the first time, it most likely enumerates a list of TCP/IP addresses from your ISP's local routing tables, and then it starts listening for new connections and sends out a request to all of those IP addresses on a specific port (1214 in Kazaa's case) to see if there are other Kazaa applications listening at those IP addresses. If so, it'll update it's user/file list with the user/file list of that computer and move on to the next one, until it eventually has a fairly complete list.
It's works in sort of the same fashion as broadcasting, except that it's using TCP/IP instead of netbios, which makes it a little more reliable. Chances are, it probably has its own host table as well, where it stores IP addresses (in a flat file or something) on your local computer of other Kazaa "Supernodes" that it's found in the past. So the next time you start it up, it'll send a request directly to those IP addresses first, and then attempt to look for others.
As more Kazaa applications start up across the Internet, the host tables in the routers will naturally keep the routes alive so other Kazaa applications can find each other more quickly and easily.
When you do a search for a specific user that's not already in your user list, it sounds like it'll go out and ask those other Kazaa Supernodes if they've heard of them until it finds one that has, which then checks to see if they're online. The request is probably passed off to other Kazaa applications as the routes get longer, and they'll continue to look for the user while your Kazaa application is simply waiting for acknowledgements.
In other words, say you're located in Los Angeles and you're looking for a user in New York. First, your application will look for the user locally, asking all of the Kazaa applications in Los Angeles first, then spreading out. Then those applications will also send out the request and where it came from (you) so it knows how to let you know when the user is found. Eventually, you'll have searched all over California, then Arizona, Nevada, and so on, until it reaches its way to New York and finds the user or times out, whichever comes first.
When that happens of course, you've just opened up a whole new world of routes to Kazaa applications for yourself and those around you, thus it won't take long before all of the Kazaa applications know of each other's existence. As it becomes more widely used, it learns of more and more applications within your area and will know how to find them quicker the next time.
Routers are set up so that they'll first attempt to route you to shorter distances (the path of least resistance) by associating costs with the lines to which they're connected.
It sounds like it would be a very long process, but we're only talking milliseconds here, considering all of the Supernodes are running with broadband connections.
If you've ever worked with domain controllers, they have a similar logic, where periodically inform the other domain controllers of users who've logged onto a domain or logged off. It's a synchronization process that occurs roughly every 15 minutes by default.
It's sort of like spreading rumors. When you connect/disconnect, you tell those around you, they tell their friends, their friends tell their friends, and so on and so on.
Another thing they may be doing is that they may actually have a centralized "server", though it's not really a server, but simply hosts a list of IP addresses that have been found to be running the Kazaa application. That would be fairly easy to do, as they can get the IP address of your computer when you download the application, sign up for the service, or start it for the first time. It may send your IP address to a host file of some sort that all of the Kazaa applications will look in to find other Kazaa applications. That would be a more efficient way to do it in terms of bandwidth usage, though they claim that there's no centralized server, although it wouldn't really be a server per se, so who knows (besides their developers).
Again, I'm just guessing at all of this, but unless they've come out with some new top-secret invention that nobody's ever heard of, and without a centralized server that you specifically tell your client to connect to, I can't imagine it being possible any other way.
As for your other question, I'm not really following what you're asking.
Anyway, hopefully someone else will intervene here and give you their two cents, and then you can go out and buy yourself a cup of coffee! No seriously, maybe if enough people get involved, we can all come up with the next best thing for you and you can make millions. Then you can buy us all a cup of joe!
Hope that helps,
Don
|
|
|
|
|
One more important thing.
I believe it would be possible to do this with the NDK, though you'll need to put some logic in there to first send a ping to an enumerated IP address (one ping), and if it's alive, then see if the application is listening on that port. If you don't get a reply from the first ping, then move on to the next one.
Otherwise, you'll experience long delays because TCP/IP by default, will naturally attempt to retry the port on a computer until it times out.
You can also specify TTL's (Time To Live) and timeouts with your ping, so you'll only attempt to connect to those who've responded within x milliseconds, and effectively those who are the shortest distance from you. After all, you don't want to establish connections with computers that have poor latency between them because it'll make your application appear slow, hence Kazaa "Supernodes" will only run on computers with high-speed Internet access.
You might want to play with a "port scanner" of some sort, write a small application that enumerates and pings ip addresses within a given subnet, and play with the ping utility itself. Typing "ping /?" at the command prompt may give you a little insight as well.
Okay enough! I have my own delima's that I should be working on!
After six long months of development, sleepless nights, beating my head against the wall, and three months of beta testing, I'm finally getting my Web Store together so I can go live tomorrow and sell my first application. Tomorrow's my big day! Then it's on to days full of answering technical support questions and dealing with a bunch of nagging customers...
|
|
|
|
|
wow..that was a long message...
(WARNING!) the following text is just a newbie view of things, and a collection newbie ideas and thoughts, so give me some credit here... (and forgive the all-mighty typo's in this and my previous posts.. )
yes, i hope someone else is going to drop into this discussion, more brains - more ideas - more productivity etc.. okay, i've done some deeper digging into the NDK source and the demo, and i think it would be possible to manage p2p (dis)connections, with a couple of new classes of course. the real thing which still bothers me is that 'user search'. the country i live in (croatia, east neighbour of italy) uses DHCP addressing system, and these people are my primary targets (my idea is to make an official messenger for the first croatian metal e-zine). so, if people have no static IPs, i suppose enumeration technique would be useless. this dynamic IP comes in handy if you're a hacker, but it's not helping me. since i'm a newbie in cpp winsock programming, and i still have a lot to learn about TCP/IP, i'm going to ask this (probably) stupid question: is it possible to search for a user over the net by looking for the COMPUTER NAME of that user? users do not change that like...every day...i guess...unless they're really weird
if that IS possible, i could talk to the webmaster and convince him to add a section to the site with registered users (i.e. they computer names). that way the community would be well known, and it would be easy (i think) to find the doode(s) you're looking for. that way it would be possible to implement a list of friends (some sort of addressbook) and/or enemies to ignore..
and yes, this is what i had in mind in the first place: app should be 'server' and 'client' at the same time. even the user status could be implemented (like all those fancy messegers, "online", "away", "not available", "toilet work" )
as for the port scanner, i basically know what it is (i know of the fact that hackers use them to seek for active trojan servers), but i have no idea how to code it (still a darn net programming newbie, and proud of it ). the idea seems simple - go from one address to another and check if my app is listening on a specific port. if it is, then maybe i could get some info about the listening user and, if i like/know him, connect.
phew. i know programming is never an easy thing to do, but all this seems kinda complicated (at least to me). i have a feeling that there SHOULD be a better concept, but who knows...
and the question you didn't quite follow is it better to create a dialog resource and class for it which would inherit e.g. CNDKClient class properties (as seen in NDK demo project) OR to use a public/private variable of type CNDKClient. darn, i can't make a good sentence... like this:
<br />
class CSomeDlg : public CDialog, public CNDKClient<br />
{<br />
...<br />
};<br />
OR something like:
<br />
class CSomeDlg : public CDialog<br />
{<br />
private:<br />
CNDKClient* m_something;<br />
....<br />
};<br />
by your experience, which one of these is easier to maintain and/or code etc..?
as a result of your typing here, i'm gonna dig into the source code of p2p programs on sourceforge, maybe that will demistify 'their concept'. if i find something usefull i will sure post it here, and meanwhile i hope someone has a new superidea on how things are done and will post it here )
in the end, allow me to shake your hand for finally finishing up thy work and i wish you the very best of luck, i'm sure you deserved it. (like you're gonna need it with those skills ) okay, back to work..oh and, you could post your site address here so we can check it out
thx
t1
---
kick ash.
http://t1tan.cjb.net
|
|
|
|
|
Well, in terms of the DHCP thing, they could be simply relying on several things like the fact that DHCP addresses are leased and expire in x number of days, and most high-speed internet access related computers are online all, if not most of the time, especially by those users who use products like Kazaa. As a result, when the lease expires, if it's even configured to expire (in rare but some cases), it's immediately leased right back to you if you're online at the time, or as soon as you turn your computer back on. Since everyone else is that subnet are most likely high-speed internet users using the same ISP you are, the same applies to them, hence when their leases expire, they typically get the same one they had before.
I had the same DHCP leased address for six months, until I went camping for a week. When I came back, someone else had gotten my lease and I received a new IP address.
But anyway, after thinking about it a little more, I don't think that's how they're doing. They COULD be doing it that way, but it would be extremely stupid.
What they're probably doing is reading/writing and FTP'ing a file with all of the necessary user and computer information in it.
Your application can connect to an FTP server, where you have the file stored, copy it down to the local workstation, read and write what it needs to, and sends it back up.
For example, if they have a "friend" user saved to their list, it would look up that user in the file and see what their current status is, and it would also update your own information, then upload it back up to the server. Meanwhile, your computer is also making connections to the users within or near your zip code (obtained from the file) AND to the "friends" that you've added to your users list to let them know that you're online now or whatever.
This way, every time someone connects, disconnects, changes their status, etc. the file will be written to, read, and manipulated for the eyes of all of your other running and connected applications to see.
There may be an even better way to do it, but it's a good start.
You can email me at dearnest@tampabay.rr.com and I'll give you more specifics on this approach.
Cheers,
Don
|
|
|
|
|
Come to think of it, there's probably two files. One with a list of information that you've saved to your local drive, and one that gets dynamically updated the way I just described.
Your application could look in the locally saved file first, then download the other file and compare the two. It could then copy new users (not already in your local file) that are close to you out of the dynamically updated file and into your local file, then copy it back up. Also, when you save a user in your list, it'll copy their info into your local file.
See where I'm going?
|
|
|
|
|
See my posts below about win2k and vc6. Hmm same error, same solution. Makes ya think huh? The finding the problem wasn't blocked some magical, mystical .net gremlin.
The original code worked because of timing. This is obvious from the fact that inserting Sleep(1000) fixed it. The code that fixed the problem protects against events that only cause an error if the events occur too often. With the multitasking nature of current operating systems, sometimes the system is doing something else which adds some time and the small window for the error to occur is masked. Each of the running programs have there own ‘frequency’ even though they are physically on the same clock. These frequencies interact in complex ways creating and masking the window. Add in the network traffic delays and you end up with a complex mess in the time domain.
Possibly .net changed some timing parameters that made the problem more obvious but it was there all along. This is a perfect example of observer error and the herd instinct. Someone screams ".net" and most people ignore the evidence and join the herd. It is easier wait for an update from Microsoft than to figure out the real problem.
I don’t really like this solution. It seems to me inelegant. The real problem, I suspect, lies deeper in the core of Microsoft’s implementation.
I give you now my 3 rules for effective debugging:
3- Never believe the user.
2- Never believe the programmer.
1- Never believe your own eyes.
|
|
|
|
|
Well, I can't speak for everyone here but I do appreciate your opinion and agree with what you're saying. Unfortunately, opinions don't pay the bills. Some of us don't have the luxury of having a lot of free time on our hands, while facing that pesky little thing called a "deadline".
My guess is that if everyone here was familiar enough with winsocks, callbacks, blocking, etc., we'd be writing our own winsock classes and wouldn't need the assistance of something like the NDK.
In conclusion, my "herd" may be following me down a less scenic path, but as long as we reach our destination successfully and take notes along the way, we'll know better next time. That's why Bill invented upgrades. My only options were to delay the project or get it on the shelf, a no brainer when you're self-employed.
|
|
|
|
|
I've been developing an application using the NDK as the communication backbone for the past six months or so. I've been scripting for several years and I've done some database development back in the day, but this is my first experience with C++, so please bare with me.
When I first started this project, all I had available to me was a copy of C++ 5 and it sounded like a job for winsocks. So, I bought some books, started investigating, found this website, and realized that much of the work had already been done and they were giving it away! All I had to do was take little bits and pieces from here and there, put them together, and voila! Piece of cake! Right.
After several long months of solitude, sleep deprevation, no social life, nearly going blind, the loss of a few friends, nearly losing my girlfriend, and nearly losing my mind, it was finally working... technically. The clients were sending and receiving, staying connected for days, taking up very little resources, the timing was excellent, and it was even tested over a sat link between Florida and Doha, Qatar, and it performed wonderfully. I only had some cosmetic issues (screen resolution stuff), and some mfc dll problems, which I later realized was due to the fact that I was compiling with "shared" dll's instead of "static".
So, I thought that upgrading and comiling with C++ 6 would solve some or maybe all of my issues, but the copy that I "acquired" would only turn out to be an Introductory Edition and could not be used to freely distribute my application.
That's when my client offered to fork over a licensed copy of .NET 2002. I'm happy to say that all of my cosmetic issues are now resolved, the installation procedure is sound, and I just sent out the first beta to a major client Monday morning.
Then, they called me Monday afternoon and said that the clients were losing their connections. I argued with him. Surely, he'd done something wrong. There couldn't have been anything wrong with the code. It was working great before and .NET is only supposed to make the world a better place!
Unfortunately, later personal tests concluded that my clients were now losing their connections in no particulure interval, order, or other consistent fashion.
So here I am. I see that others are having the same issue and I don't know whether I should be happy or miserable about that, but I know that I just took a huge step backwards and I'm passed my deadline, I haven't slept in three days and I've scratched my head to the point of drawing blood, and I'm in desperate need of outside assistance!
Here's the only difference that I can think of, maybe a clue. When I attempted to compile the first time with .NET, it complained about a line in NDKUserManager.cpp:
CNDKUser& user = m_users.GetNext(pos);
The error was:
error C2440: 'initializing': cannot convert from 'const CNDKUser' to 'CNDKUser &'
I changed it to the following and the compile error went away:
CNDKUser user = m_users.GetNext(pos);
Thanks for any help in advance,
Don
|
|
|
|
|
Hi,
You're right about changing the code "CNDKUser& user = m_users.GetNext(pos);"
to "CNDKUser user = m_users.GetNext(pos);" in order to remove the compiler error BUT the problem is not there...
I can guarantee that the problem is not related to the NDK itself because it's only an architecture over the CSocket and CArchive. So the problem comes from CSocket and/or CArchive.
Someone else found an article from Microsoft with a possible problem
http://support.microsoft.com/default.aspx?scid=kb;en-us;185728
"In Windows Sockets, you should not make multiple recv calls within an FD_READ notification unless you are willing to disable FD_READ notifications prior to calling recv. However, CSocket and CAsyncSocket make no provision for doing so. Therefore, you should make only one Receive call per OnReceive function. Under high data transmission rate, if you make more than one Receive call in the OnReceive function, the application might lose FD_READ, have fake FD_READ, or have no FD_READ (hanging)." "You can use CSocket with CArchive and CSocketFile to directly receive and send MFC CObject-derived objects. However, under high data transmission rates, you should not use CSocket with CArchive and CSocketFile within the OnReceive function because they might internally generate multiple Receive calls."
I suggest to you to visit MSDN Newsgroup.
If you find something PLEASE tell it to us.
Thank you
Sébastien Lachance
|
|
|
|
|
Hi Sebastien,
You'd already replied by the time I wrote Part 2 of my delima! Thanks for getting back to me so quickly, and thanks for the NDK.
So, is this a problem with the included CSocket and CArchive files that came with .NET then? Is it safe to say that this problem was introduced in .NET? Could I simply replace these files with the 5.0 files? If not, could I go back and recompile the code in 5.0? Would that make these problems go away?
If this is in fact a Microsoft issue, I don't really have time to wait for them to resolve it, so I'd rather get it working for now and upgrade it later. Again, I'm fairly new to C++. I noticed that others are having this issue outside of the NDK but there doesn't seem to be any resolution.
I'll keep digging, but do you think stepping back to 5.0 will solve the problem?
Thanks again,
Don
|
|
|
|
|
I cannot say that the problem comes from entirely with .NET but it seems that I receive many mails since that.
I don't have 5.0 anymore.
If you have it, you could test it easily. You only have to create a workspace then insert .cpp/.h files.
|
|
|
|
|
I recompiled with VC++ 5.0 and everything is working perfectly again. I even took out all of the client heartbeats and they've been running and staying connected for going on several hours. I'll check back in a few days and send an update, but it looks like this solved my problem and may solve others.
I still have the PingAllUsers on a timer in the server, just in case a client crashes, and that's working great as well.
Maybe the 5.0 winsock files could be copied into the include directories of .NET and after recompiling, similar problems would go away, but as long as it's working this way, I'm happy with the known good. Lost connections after days I can deal with, but not minutes!
I guess the moral of this story is to be cautious of the "latest and greatest"!
Thanks again for you help!
|
|
|
|
|
Good! I'w waiting for your confirmation.
Sébastien
|
|
|
|
|
Still cooking with gas!
Since I recompiled everything with 5.0, I haven't had one single problem. The clients have been connected non-stop for the last few days without failure and no messages have been lost lost.
In fact, the application will be beta tested on three different WAN's, including satellite links between Florida and the middle east and ranging from 500-1500 clients starting next week. I'm just polishing up the documentation.
Has anyone performed any stress tests? I'm curious to know if there are any limitations on the number of clients that a server can handle and what might happen when that threshold is reached. I'm sure it wouldn't have anything to do with the NDK itself and that the limitations would most likely be those of the hardware or OS, but if anyone has tested a similar client/server application with this code on a large network, I'd love to hear the results.
This way, at least I'll know there's a problem BEFORE the beta testers break it and start complaining!
Cheers,
Don
Great class by the way. Thank you very much! You trimmed off of my project what would've probably turned into months of frustration!
|
|
|
|
|
Check just below... http://www.codeproject.com/internet/ndk.asp?select=506839&df=100&forumid=1156&fr=26#xx522006xx
Just disable the read notifications, and re-enable it after.
Djof
"Je me souviens"
|
|
|
|
|
Well done!!
Can you confirm it really works?
Thanx
Maurizio
|
|
|
|
|