|
Another day, another bunch of code fixes.
The latest update today is mainly concerned with stability and with enhancements to the Job Board and Message system.
1. Stability.
Our error tracking reporting system has been much improved but our error rate has declined to the point where we don't actually get to use it that much. The delicious irony. Most of the errors we do see are the occasional null ref error, a view state error or an internal value check error that goes along the lines of "OMG!! The value was less than 0!!1!" which we ignore because the system will handle this correctly anyway, and the cause of such values is usually a malformed URL.
2. Job Board.
More tweaks to the layout, to features, and to the explanations of things like Coupons. Lots of additions to admin on the backend, which is probably of no interest to anyone reading this, but it certainly makes our life easier.
3. Message Boards
The drop-down menus are now more sensible but I've still yet to fix up the message board listings in the pages themselves. At the moment they are a little random.
The other big change, apart from a little optimisation and stability, are the big buttons that allow you to say that an answer to your question was helpful. The reason for this will be clear as soon as we have completed one last piece of the puzzle. The puzzle being "Why is SQL using an index scan instead of an index seek when it's happy to use the index seek when the query is executed directly but not when the query is inside a stored procedure".
Oh yes, our dinner parties are full of crazy exciting talk.
cheers,
Chris Maunder
CodeProject.com : C++ MVP
|
|
|
|
|
One of the hardest things to do when developing software is to look at your creation from with the eyes of a newcomer. Does it make sense? Is it overwhelming? Can I work out how to use it? Can I find stuff?
I had one of those moments a while ago while creating a Press Release[^] and realised that it was difficult to track the growing number of features we're adding to The Code Project. As an author you want to have access to your list of articles. As a user looking for answers to questions you want to keep track of the messages you have posted. As a company posting a Company Profile[^], and maybe a press release you want to keep track of these, and as an advertiser you want to get access to the latest stats on your ads.
So My CodeProject [^]was born. A single page that lists those items that relate directly to you. Forums, articles and authors you want to watch; your company listings and press releases, links to your messages and your articles, and a ton more to come.
The page itself auto-refreshes by default every 15 mins (though you can change this easily enough) and will stop refreshing when it stops seeing mouse movements on the page. There's no point in refreshing a page if no one is there to watch it.
The next step will be more information on forum participation, then a few bits and peices based on a new service we'll be rolling out soon. Obviously more customisation and personalisation is a must, but we're always open for suggestions and would love to hear what you'd like to see on that page.
cheers,
Chris Maunder
CodeProject.com : C++ MVP
|
|
|
|
|
One of the things we've been focussing on is storing and displaying more information on what's been going on around the site. Page views, voting history and downloads are 3 major areas we've been working on and as a test we're now exposing page view graphs for members who are gold and above. All they need to do is visit their articles and scroll to the bottom to get a technicolour (well, orange and gray) view of just how popular they are.
Maybe it's useful, maybe it isn't, but it doesn't hurt to give these things a shot.
cheers,
Chris Maunder
CodeProject.com : C++ MVP
|
|
|
|
|
BTW - what's the "watch article" tool do? Send out emails when the article is updated?
Citizen 20.1.01 'The question is,' said Humpty Dumpty, 'which is to be master - that's all.'
|
|
|
|
|
I've had a couple of questions about the membership level system and why levels have changed.
First we had the 'Woohoo! Platinum!' bug which, we're sorry to say, we had to fix. Actually it was a member-wide bug that gave everyone, for a few, happy days, 1 level higher than they were.
Then we made another change which sent a lot of Gold members back to Silver. We did this because we needed a way to reward members for participation and not just for being a member a long time. The issue we face is that we are trying to hand off more and more of the policing around CodeProject to the community so we need a reasonably large group of members who are active and whom we can trust to act responsibly. We thought that Gold members would be perfect for this but we found there were a lot of gold level members who were gold merely because they'd signed up a couple of years ago and were misusing the trust we'd placed in them.
We rejigged the system so that Gold required participation and it's since worked out really well.
Overall I feel it was the correct thing to do because the member level system should not be simply a reward for not wandering off into the sunset. It should be recognition of participation.
cheers,
Chris Maunder
CodeProject.com : C++ MVP
|
|
|
|
|
We've made a bunch of changes, some useful, some mere eye candy that actually just lessen the load on the servers (think Ajax), and some you will not even be aware of.
Here's a quick rundown of what's been going on in the last week or so:
- The vote histogram (old news, but something I've wanted forever)
- Ajax voting on articles
- Ability to delete/rename files associated with your articles
- The simplified question/answer voting in the forums. We'll be extending this in the next couple of weeks
- Replicated database servers. Replicated with our own special sauce, though, so there isn't the usual lag problem that replicated database servers have. Added to this is database monitoring tied directly into the webservers. Speed, redundancy and automatic failover. All I can say is woot.
- The usual quick fixes left, right and centre.
- The new sites! Having the ability to partition out our data into new sites was a design goal of the rewrite and we can now have a sub-site up and running in minutes. We want to expand our offerings but we want developers to feel they have a home of their own.
- Tables of Contents now have a simple article filter. Just start typing and articles that don't match the text will be removed from the listing. Still a few improvements to do on this.
- The article moderation system. This seems to be serving its purpose nicely but the side effects (highlighting the bad articles) has been an interesting learning experience.
- RSS feeds on all programming forums.
We'll see what we can rustle up next week
cheers,
Chris Maunder
CodeProject.com : C++ MVP
|
|
|
|
|
Chris Maunder wrote: Ajax voting on articles
Yay!
I see you changed the look of profile forums a bit. Hope this goes through...
Citizen 20.1.01 'The question is,' said Humpty Dumpty, 'which is to be master - that's all.'
|
|
|
|
|
The biggest problem with a site that churns out an enormous amount of data is in ensuring that a user finds exactly the information they need. There are a number of ways in which this can be done:
1. A good search engine tuned to the needs of your users
The standard SQL Server full text search does not handle "C++" very well so care must be taken to ensure that you do some magic to handle specific cases like this. Further, providing ways to specifically limit searches (eg filter by attributes) will mean that search results are filtered, and not merely ordered by, the attributes of the content the user is after.
2. A well planned Taxonomy and guidance trails
Breaking the content up into Chapters, sections and subsections that make sense and will allow the majority of users to intuitively find the material they are after is not as important as a good search engine (Google has trained us well) but is important once similar content to the target content has been found. Provide the means to trace back up a level (or more) so the user can then find material near the material they may have stumbled upon
3. Provide the means for your users to promote material
While ratings and reviews are inherently subjective and open to abuse they are invaluable in allowing a user to determine within an order of magnitude of confidence which of the 10 results they are looking at they should investigate first. One developer's gem could be another developer's "Not Another Article On..." cry of frustration. It really depends on which article the developer found first, and in this respect ratings are even more subjective in that they are not only audience-dependant but also time dependant. Two articles on the same topic immediately one after the other will have a very different voting pattern to the two articles posted months apart.
The way in which a rating is displayed is extremely important as well. A straight number doesn't mean anything. A number that relates to the number of times an article has been rated, as well as it's actual rating provide more feel for what the general populace feels, and a rating that provides information on the distribution of the ratings provides yet more information, since ratings well outside a standard deviation show up and can be mentally discarded by the reader.
Guiding the way in which your readers vote will allow you to use the rating values more effectively. Previously we used the "Vote 1 to 5" method for all posts indiscriminately but we've not moved to a "Good Question/Bad Question", "Useful Answer / Unhelpful Answer" options that makes it clear as to the feedback we are after.
4. Feed the data back into the system
Once you have a good search engine and taxonomy you need to feedback the work your readers have done into your system. Our search results are based not just on keyword relevance but on the popularity of an article, and at the bottom of each article we provide lists of similar top rated articles. Our Good Question / Bad Question system will (hopefully) train members to either form their questions better, or ignore questions that are posed by those unwilling to show any initiative themselves. We can then ensure that the better-posed questions will get more of the attention of members looking to help out.
cheers,
Chris Maunder
CodeProject.com : C++ MVP
|
|
|
|
|
One thing I've been trying to find the time to do is write up this whole rewrite thing we've been doing. Unfortunately we've all been doing silly hours with our little trial by fire but I do need to get the ball rolling:
some whacky and amusing things we found:
- SQL Server, IIS and the ASP.NET framework are possessed.
- Using development settings instead of production settings is a Bad Thing.
- Storing session state in a SQL database can be a performance killer. Especially when the database is an old P4 test harness machine 40 km away from the webserver. That was an interesting experience in scream therapy.
- Always check your indexes. Especially the ones that absolutely positively are not and cannot be fragmented. They will be the ones most in need of attention.
- 20,000 users hitting your application is wonderful load testing. We did load test but our testing machines did not generate the load you guys can generate
- no matter how many ways you check code there's always an angle that you haven't looked at. We're finding errors in 10 mins of being on that we couldn't find from 3 months of beta testing. This is just nature.
- Do not take error message reported by the .NET framework at face value. For instance: a SQL command timeout can actually be caused by a web server timeout. Strange but true and it caused us no end of frustration
cheers,
Chris Maunder
CodeProject.com : C++ MVP
|
|
|
|
|
Chris Maunder wrote: Using development settings instead of production settings is a Bad Thing.
Very True. And sometimes this would invite security vulnerabilities into the application also. Isn't it?
Chris Maunder wrote: - 20,000 users hitting your application is wonderful load testing. We did load test but our testing machines did not generate the load you guys can generate
- no matter how many ways you check code there's always an angle that you haven't looked at. We're finding errors in 10 mins of being on that we couldn't find from 3 months of beta testing. This is just nature.
I agree. At least from India, just fresh graduates are recruited as testers. The management says it is cost-effective to take fresh graduates but it is a pain to train them into our applications and then they 'air papers ' out. And these people know about clicking, double-clicking and re-clicking the same link over every release of the product. The true users report functional errors and these cabbage-testers are just fit to archaic testing only to report to their project managers of more activity in their aghast objective of appeasing thier ass.
Chris Maunder wrote: - Do not take error message reported by the .NET framework at face value. For instance: a SQL command timeout can actually be caused by a web server timeout. Strange but true and it caused us no end of frustration
If you could illustrate this and explain a bit more, it would be of immense use for many developers.
Vasudevan Deepak Kumar
Personal Homepage Tech Gossips
A pessimist sees only the dark side of the clouds, and mopes; a philosopher sees both sides, and shrugs; an optimist doesn't see the clouds at all - he's walking on them. --Leonard Louis Levinson
modified on Friday, January 04, 2008 7:12:01 AM
|
|
|
|
|
can you help me for Read this address < 0xFEBFFC00 >
0xFEBFFC00-0xFEBFFFFF Intel(R) 82801EB USB2 Enhanced Host Controller - 24DD OK
this address in ( System Information XP \ Hardware Resources \ Memory )
|
|
|
|
|
It says: "You should have hung a left at albuquerque".
|
|
|
|
|
It's been ages since I've provided any type of update on what we actually do in this office of ours.
Mostly fighting with the guys across the hall as to who owns the espresso machine, but that's another story.
Some background:
The Code Project was not planned or architected or written in the usual "corret" way. It was built bit by bit as time, resources, bandwidth and server power allowed. It was also built on VBScript and ASP so it can (now) be considered antiquated and backwards.
Yet it still works. Quite well, usually.
In June 2000 Microsoft announced .NET and I immediately started rewriting the infrastructure. Partly to take advantage of the promised speed and efficiency, partly to learn the new technology. Unfortunately at the time resources (ie warm bodies in the office) were scarce, the .NET framework wasn't mature (actually, it wasn't even released) and growing and caring for the site became more important than playing with new toys.
Over the next couple of years as .NET 1.0 and then 1.1 were released various incarnations of the underlying Code Project framework were written. Increasing load on the servers and on us personally meant we had to make quick, and sometimes painful decisions as to what we would do. Employ someone for 80K a year to rewrite the infrastructure while I worked on keeping the community growing, the articles published, the ads that keep us alive fresh and exciting, the bills paid and the coffee warm and aromatic? Or just buy a few big whopping servers that could better handle the load?
We chose the latter which meant the site could continue on without interuption and I would be free to make optimisations at a more sane pace. The main infrastructure stayed ASP/VBScript and most of the backend caching, support and advertising system was using the new .NET framework. Yep - we've been .NET for a long time.
However, there comes a point that even the best (or luckiest) organic architecture starts to reveal that it's not up to the task and so last year we finally decided to throw everything away and start fresh. The goal was speed, efficiency, low error count, scalability and extensibility and, #1, more developers understanding the code so that I could have my weekends back.
Dmitry was hired to completely rework everything SQL, and soon after Elina was hired to handle the business and UI layer. Christian Graus then came on board to work on one of the first plug-in modules for the site.
Based on my initial designs, my tales of woe, the constraints we faced and the goals we had for the community we soon had a basic infrastructure built in .NET 2.0 running on SQL Server 2005. Scalable, web farm happy, extendible, solid and way more efficient than the old VBScript code.
For the 6 months since then it's been a matter of going through the site and replicating and improving the functionality. We're almost there with only a couple of modules left to go.
The interesting thing in all this has been the things that have held us up.- Coding and naming standards have helped enormously, even when they have changed and forced us to take 2 steps back to rework code.
- Moving to a completely new database schema has mean an entire conversion module has been written to ensure not too much breaks.
- Ensuring we take no shortcuts so that it's easy to create a mobile friendly and cross browser friendly site
- Always looking at how a stored procedure, or a constructor, or a caching decision will affect performance
- Always considering the direction we wish to extend the codebase. Have we backed ourselves into any corners
- Code Reviews, FxCop reviews, Unit tests, walk-throughs, and the very worst: really cool new ideas that we must decide whether to implement now or later.
- Synchronising changes to the current ASP site and ensuring they are included in the ASP.NET functionality.
But mostly it's been about ensuring the current site gets as much love and attention as possible while devoting as much spare time as possible to the rewrite. A week can go by in which that small 2 hour job falls by the wayside due to demands from the site which can be incredibly frustrating.
One step at a time.
cheers,
Chris Maunder
CodeProject.com : C++ MVP
|
|
|
|
|
Hi Chris,
how about enabling the codeproject to get posts from Windows Live Writer?
A xml API.
This would be a great enhancement. What do you think about this idea?
Regards,
Rainer
Live is to short to be angered about lost chances!
|
|
|
|
|
Unrelated to this reply..
With regard to: Timestamp program.
Wouldn't it make more sense to change the offset from the current modified timestamp.. and not the offset of the current computer time.
Example: I fix a few files and need to send them to someone in a different timezone so that they can see when it was modified in their timezone.
I fix them but am not prepared to send them the moment I fix them.. or I forget to send them right away.
Currently the offset is relative to current computer time.. when in fact to be more accurate it should read the existing modified timestamp and apply the offset to this.
I am not a programmer so it is not easily known to me how to make this change in your program.
Perhaps you could enlighten me.
James De Lucia
James@TriDeLsolutions.com
|
|
|
|
|
This in itself would be worth writing an article about to make the realities of the world clear to those that think that it is not worth trying to optimize code (or write optimal code in the first place).
Peace!
-=- James Please rate this message - let me know if I helped or not!<HR> If you think it costs a lot to do it right, just wait until you find out how much it costs to do it wrong! Avoid driving a vehicle taller than you and remember that Professional Driver on Closed Course does not mean your Dumb Ass on a Public Road! See DeleteFXPFiles
|
|
|
|
|
all the hard work is paying up now. CP looks great. the speed that you have been talking about is now felt with all the added cool functionalities. congratulations and thanks for providing us with Code Project!! now where's that case of beer? we might need that soon.
|
|
|
|
|
I don't want toys or new shiny things. I don't want Visual Studio Orcas or Windows Vista Server. I don't even want service packs.
I want versions of the Windows applications that I must use in my day to day work to be optimised. To be made faster. To not use so many system resources, so much memory, so much disk accessing and so many network roundtrips that they are rendered unusable while your machine digests their request.
SQL Server management console is currently the poster child (for me) of an application developed by someone who had the server in the room next door to them. Try bringing up a context menu on a server across a slow connection.
IE is the embodiment of inefficiency bought on by a "we'll use the same function for everything" mentality. Click on a new tab and you get a "Connecting..." prompt and enough time for a quick lie-down. Just bring me up a new window. Don't connect to anything. Don't load anything. Just the window please. Firefox does this task instantly.
Outlook is a great example where they forgot, in places, that there was a real live human trying to use the application. I typically read emails in the preview pane and then right-click to mark as read (or leave it be if I need to follow up). Sometimes I make the horrible mistake of right-click and selecting the option underneath, namely "Categories...". This can lock up outlook for minutes. Where's the multithreading? Where's the "cancel this incredibly slow operation you didn't mean to do" button?
Also: I get thousands and thousands of spam a day and they get kicked into my spam folder at the exchange server level. If I try and empty this folder I can't do anything else with Outlook. It's blocked.
Please: PLEASE do some housecleaning on Outlook for us poor souls who rely on it to handle thousands of emails.
I'm not even going to discuss Visual Source Safe. Microsoft: please just buy a new solution that works. Ionforge Evolution works like greased lightning for me across 14,000km (Australia to Toronto). Visual Source safe is unusable when outside the LAN.
Visual Studio 2005 is the final splinter under my thumb. Refactoring is not a feature anymore, it's a necessity. The refactoring in Visual Studio is so incredibly inefficient that it's essentially unusable. When you do find you have to use it it's always with that "Oh God no" feeling.
Similarly, Building a solution. Builds are a really, really old technology and the problem of efficiently building applications was addresses - I dunno - over 20 years ago? Why then is it that everytime I do a Build, Visual Studio takes such an inordinate amount of time to determine which projects should be rebuilt and which can be completely ignored.
We could all go on and on and on. I'm not angry, I'm just disappointed. What happened to pride in applications? To being careful with resources? To being clever with the way you do things so you save cycles or, even better, network roundtrips, unnecessary function calls and minimise disk access.
So here's what I really want: I want a new "VP Usability" job position at Microsoft. That person would make sure the applications had a consistent user interface, would ensure applications were simple to use, played nice with your computer, and were snappy. He would be given a 2GHz Core Duo laptop with 1Gb RAM and a 5400rpm HDD. He would attend all meetings with that and his swearing and cursing recorded diligently. He would berate sloppy, corner cutting programming and reward, with riches beyond their wildest dreams, exceptional programming. He would have style and class and absolutely zero patience.
And his first job would be to fire the entire WMP team.
cheers,
Chris Maunder
CodeProject.com : C++ MVP
|
|
|
|
|
I think we are seeing this in many applications. Astronaut architected "platforms" masquerading as applications. Outlook is not so much an email tool as an information broker platform these days. I'd prefer if they made it an email tool again. Firefox is OK now but the next version is going to have all sorts of "application platform" aspirations and I fear it will revert to the bloat that was Mozilla.
Great, it can do X, Y AND Z but guess what, it doesn't do A very well anymore and A is 90% of its purpose. It integrates with G? Great! But I don't have G. It makes me coffee? Fabulous! Except this other dedicated coffee maker makes better coffee so I'll stick with that, thanks.
There is a legacy of "features/capabilities == competitive" thinking. People think it is difficult to release a focused application that does one thing well. They fear the initial volley of blog/magazine/pundit remarks along the lines of "Those idiots forgot Feature K!" which does happen but, with good tools, is often followed by a dedicated group of users that are worth more than the billion use-once-never-again users.
Flash in the pan > long term effect.
And as much as I prefer Mac OS X to Windows these days, it is not perfect either.
It is difficult battling the Feature Demons though. You can be as rational as gravity but you'll come out battered by feature requests you know are superfluous and ultimately damaging to your users productivity. Too many are afraid of making a stand, of forging a path. Yet we have seen that those who do focus win out in the long run.
I don't think firing people is necessary but Microsoft in particular have lost vision, have lost a champion at the front to drive them forward. Apple are doing well off of Jobs and I'd like to see Microsoft, which has a much tougher challenge than Apple, find that person and vision again.
regards,
Paul Watson
Ireland & South Africa
Shog9 wrote: And with that, Paul closed his browser, sipped his herbal tea, fixed the flower in his hair, and smiled brightly at the multitude of cute, furry animals flocking around the grassy hillside where he sat coding Ruby on his Mac...
|
|
|
|
|
Chris Maunder wrote: Please: PLEASE do some housecleaning on Outlook for us poor souls who rely on it to handle thousands of emails.
pfft, try managing 1 email in Notes, then you will know real pain. Or even worse, just logging into Notes.
|
|
|
|
|
I can't agree with you more. Microsoft apps are not among the ones I use anymore:
- IE, no: Firefox
- Word, Excel, no: StarOffice
- Outlook, no: GMail
and the list could go on.
And let's face it, they keep Visual Studio because it's their API.
It's true: Microsoft is falling behind.
|
|
|
|
|
Clearly something's been happening behind the scenes so I figured a quick rundown of the main bits would make for interesting lunch time rading. Or something.
So here's a quick rundown of where we are:
After more than a year of upgrading and troubleshooting hardware we are at a stage where hardware is solid, our network topology sensible and our load bearable. We haven't bought a new server in months yet site performance is consistent. We are at a position where there is no need to push upgrades or increase capacity which means it is a perfect time to push upgrades and increase capacity. And because we now have the luxury of time we've gone back to basics and started from the ground up.
Every database, table and field in our SQL layer has been examined and refactored. Several hundred changes later and we now have a schema that brings together all the disparate modules that have been developed for CodeProject over time into something unified and extensible yet still modular. Strict attention was paid to keys, indexes, referential integrity, normalisation - even sensible names that reduce the chances of misinterpretations.
The next (and current) stage is taking a birds eye view of what the data layers are meant to provide the business layers, make realistic estimates on table sizes, result sizes and frequency of calling, and from there rework the queries and indexes to be optimised, overall, for our patterns of use. Some long running complicated queries are left alone since they are called once every 6 hours, whereas other queries are optimised and reworked in a slightly convoluted in order to eek uot more performance on something that is called constantly. In conjunction with all this judicious use of caching can do miracles.
Which then leads to the business layer which at the moment is confined to internal pages and applications. The data layer, base web and base standalone application frameworks are done. The CodeProject you see today is part dynamic ASP and part pre-cached data generated by backend services using the new CodeProject .NET framework. View the latest articles and the RSS feeds of the latest articles and you will, unfortunately, still see a difference in what one produces over the other.
For the most part current work on the business layer is confined to those parts required by our utilities. Having the ability to use this code in the web application as well means when it comes time to complete the various modules that make up CodeProject, 80% of the code will already be there. It'll be lego programming at its best.
The update to the presentation layer has been something we've been putting off for, well, ever, but with the speed of development increasing we found time to get this part done. The whole redesign has been discussed ad neauseum but on the whole we're pretty happy with it and happy with the feedback. It ties together the mishmash of different styles, gives us more room to move and, most importantly, works with the HTML generated by the new back-end systems.
As to what's next? We're not sure. We've got our laundry list of items to work through but there's definitely a spring in our step after such a long period of simply doing maintanence. New features in the forums and new features for our members are just too much fun to ignore so we may even delay some parts of our main work to take a side trip and throw new stuff out into the wild. With so much work being done on the backend it's hard not to want to let some of it pop its head into view a little prematurely.
cheers,
Chris Maunder
CodeProject.com : C++ MVP
|
|
|
|
|
That's great news, (little late i know) don't forget to sleep.
Brad
Australian
- Christian Graus on "Best books for VBscript"
A big thick one, so you can whack yourself on the head with it.
|
|
|
|
|
Bradml wrote: don't forget to sleep.
Yes, he needs a good deal of rest, "I call it sloth day " in which I just sleep, eat watch movies and never step out of the house!!
All generalizations are wrong, including this one!
(\ /)
(O.o)
(><)
|
|
|
|
|
There must be something fundamentally unique about the placement of Toronto because the number of bizarre, almost biblical plagues and curses that hit this town seems to me to be way above the statistical norm.
SARS, blackouts, more SARS, West Nile Virus, roads washing away, intersections caving in under sink holes, Bird Flu, the regular procession and recession of glaciers, winter smog, summer fog, debilitating cold, insanity producing humidity and I'm sure, any day now, rabid carnivorous squirrel zombies.
OK, so I made up the bit about glaciers in Toronto but it's so bizarre to me that not a day goes by without something, somewhere, being terribly wrong around here.
The latest? Alergies.
Sure, many people get the sniffles every spring or Autumn but it seems everywhere I look Torontonians have collectively turned into a drooling army of puffy eyed, coarse throated snifflers because the nervous and opportunistic vegetation and fungi have taken a peek at the sky and said "Make the most of it guys - we've got about 2 weeks before we're all snap frozen".
6 years on and Toronto still never ceases to surprise and baffle me.
cheers,
Chris Maunder
CodeProject.com : C++ MVP
|
|
|
|
|