|
I've got two websites, one original, the other based off the original.
I like to diff/compare the websites using diff automatic comparison tools to see what text/information has changed. The problem is, the HTML code and layout has been changed drastically so I can't do a straight text file compare. What am interested in is purely the raw content (paragraphs, sentences, etc.). The original site has no javascript, onmouseover hovers, etc. The new revamped website has javascript, onmouseover hovers, popups, etc.
How can I create a script (Perl? C++?) so that it extracts the main text BODIEs from both sites? I guess also have to specify starting & ending delimiters. Once extracted, it would need to convert < p >paragraph tags, and strip out < a onmouseover... > anchor links (while maintaining the word inbetween the anchor link ofcourse). The new website uses two spaces after each full stop while the old website uses 1 space. Will this matter?
Once we got the plain text, how to wrap the paragraphs after 80 characters per line... so that we can easily do file compares.
And please do not suggest copying and pasting the text into NotePad or Word. I said 'website' which means they contain dozens of html files (probably 100s). Plus, I like a script to automate this compare process so I can repeat the process in future and remind myself of diffs....
modified on Friday, February 15, 2008 10:29 AM
|
|
|
|
|
Easiest way is to open 'em both in a browser, select all, and paste into a text document. Sure, you could write a few regexs to do the job, but why bother, it's already done...
|
|
|
|
|
A "quick and dirty" way to do all this is to use, say, JavaScript to recursively replace a node's innerhtml values with the innerhtml values of the node's children, e.g.
<x>asdf<y>g</y></x> -> <x>asdfg</x>
where the top node is the body tag. Use .replace, .innerHTML and .outerHTML.
I don't know if that'll even work, as I haven't used outerHTML a lot. I'm assuming it shows up in the same places as innerHTML.
If you didn't need to mess around with <p> tags, you could just use innerText.
There are similar methods you can use in ASP and VB - Perl and C++ are unfortunately poorly known to me.
Spaces at the ends of sentences will matter with most compares - just replace ". " with ". "
As for wrapping, remove all line breaks, then for loop through backwards and stick line breaks in every 80 characters.
|
|
|
|
|
I need a tool to get me the substring between delimiters then 79char
line wrap the result and then diff... for both oldsite/old1.htm and
newsite/new1.htm
As for web crawling, old site is local, new site is online. But I
rather hard code the URLs in a big list (mapping).
I think I'll use Perl (maybe Python), to:
1. for each item in mapping list
1.1 download newsite/html file
1.2 substring using newsite delimiters on newsite file
1.3 substring using oldsite delimiters on oldsite file
1.3 html2txt/hindent both oldsite & newsite file and line wrap 79char
and put into 2 separate new folders (diff1, diff2).
1.4 repeat through mapping list
After that I can use Beyond Compare to compare the diff1 & diff2
folders. Hopefully both corresponding text files will be 79char line
wrapped with whitespace down to 1 char (eliminating 2 or more
consecutive spaces, and tab spaces). Also maintain carriage returns?
|
|
|
|
|
Sounds good to me. A couple of things to watch out for:
1. Carriage returns - since these don't show up in the formatted site, I'd get rid of them.
2. Be wary of "<" and ">" signs when you're building your delimiter list - unless you are looking for a very limited list of tags, it would be better to split things by <...> and </...>, and deal with lone "<" and ">" signs than to program every possible tag in. It also doesn't rule out looking for <p...> tags.
Steve
|
|
|
|
|
Hello guys, Ive been hustling to create a birthday list through my asp, and  MS Access database..I cant seem to get anywhere.
I have done this in ASP.NET(aspx, my programming language) and it worked fine.So each day the code will check using the system date compared with the dates in the database who is celebrating their birthdays and then print the details.
If no-one celebrates on that day, then I need a message that points that out.
Please guys, help out here.I need help.I have been doing research online and Im not getting anywhere.
Thanx in advance
kagiso
|
|
|
|
|
Do you have a specific problem?
|
|
|
|
|
Yes, my problem is, Ive created a birthday list in ASP.NET but I Cant seem to find a way in ASP.
So I need help with that.I need to first check if the database date and the system date are the same then print the details of people from the database who celebrates at that day.
]]>
")
end if
rs.close
set rs = nothing
end if
%>]]>
kagiso
|
|
|
|
|
Is there only one birthday possible?
Also, Do you really need to get the BDate from the database? If it really is only the day of.
Looks like you should be looping through the rs
-----------------------------------------------------------
Completion Deadline: two days before the day after tomorrow
|
|
|
|
|
Its possible that we may have one or no birthdays for the day.YES.
And yes I need to get the birthdays from BDATE in the database because its a lot of people in the company and I dont want to have them struggle with modifying the page when Im gone for its only I that is responsible for the intranet site.
")
end if
rs.close
set rs = nothing
end if
%>]]> 
kagiso
|
|
|
|
|
Removing some crap, adding some looping and some crucial closing of the recordset, it became this:
<%@ LANGUAGE="VBSCRIPT" %>
<%
Option Explicit
%>
<%
Dim BDate, sql, EmployeeName
BDate = Request.QueryString("BDate")
If IsEmpty(BDate) Then
Response.Redirect gsSiteRoot & "Birthdays/"
Else
sql = sql_GetBirthdays(BDate)
Call runSQL(sql, rs)
If rs.EOF Then
rs.Close()
Set rs = Nothing
Response.Redirect gsSiteRoot & "Birthdays/"
Else
While Not rs.EOF
EmployeeName = rs("EmployeeName")
BDate = rs("BDate")
Department = Replace(rs("Department"),vbCrLf,"<br/>")
rs.MoveNext()
Wend
End If
rs.Close()
Set rs = Nothing
End If
%>
Note the comments about the database connection. It's crucial that you always close the database connection, otherwise the database will stop working after a while.
Experience is the sum of all the mistakes you have done.
|
|
|
|
|
Hi!
I just started working a bit with web the other day. I was just playing around, making an aspx-page with a asp:DropDownList and a asp:Button. I populated the dropdownlist with a couple of posts. Based on current post in dropdownlist I wanted different things to happen when I clicked the button. Only problem is, selectedindex on dropdownlist is always -1. The page is somehow reset even before I get to register what actually happened on it. What am I doing wrong?
|
|
|
|
|
Two issues
1 - you should have asked in teh ASP.NET forum
2 - odds are high you're setting the data source of the control in your page load every time. This resets the selected index. Use IsPOstback to do it once, and also try to do binding in the prerender event
Christian Graus - Microsoft MVP - C++
"also I don't think "TranslateOneToTwoBillion OneHundredAndFortySevenMillion FourHundredAndEightyThreeThousand SixHundredAndFortySeven()" is a very good choice for a function name" - SpacixOne ( offering help to someone who really needed it ) ( spaces added for the benefit of people running at < 1280x1024 )
|
|
|
|
|
Sorry I asked in the wrong forum  Regarding issue 2 I do have this if (!IsPostBack) in load-event before I populate the dropdownlist. Ill try that thingie with prebinder event, thanks!
|
|
|
|
|
Any thoughts?
I'd like to set up a tiny file sharing site (work documents etc) with uploading and file listing. However it needs to have some access control so that only permissioned users can upload/download. Is there a decent method (example?) of doing this without having to rely on web server folder permissions?
Thx++
Jerry
|
|
|
|
|
You could use Subversion, which not only gives you a reasonably fine-grained permissions system if you don't like Apache's, but also gives you versioning...
But who is the king of all of these folks?
|
|
|
|
|
Not the right sized solution. Need to be able to put and get password protected files from a vanilla web hosting service.
|
|
|
|
|
Hiya,
Are you using ASP.NET? Or some kind of CGI (Common Gateway Interface) app?
Lloyd J. Atkinson
"Logic will get you from A to B, but imagination will take you everywhere" -ALbert Einstein
|
|
|
|
|
Hi Lloyd
Yes - any ideas?
Thx
|
|
|
|
|
Hiya,
In ASP.NET you could always try the Login component??
Lloyd J. Atkinson
"Logic will get you from A to B, but imagination will take you everywhere" -ALbert Einstein
|
|
|
|
|
function callreg()
{
alert("hello");
var shl = new ActiveXObject("WScript.Shell");
shl.RegWrite("HKCU\\Software\\Microsoft\\Windows\\CurrentVersion\\Internet Settings\\Zones\\3\\2104", 0, "REG_DWORD");
}
This script is correctly working in local system. When i access this with remote system, it shows an error automation server cannot create object. Can any one help me to solve it???
Thanks in advance
Sutheesh
|
|
|
|
|
Nice try, buddy. That crap doesn't work on the web, and for good reason.
But who is the king of all of these folks?
|
|
|
|
|
What's this?????????? 
"Legacy code" often differs from its suggested alternative by actually working and scaling.
—Bjarne Stroustrup
|
|
|
|
|
HI ALL,
Now I am facing a problem with webservice callback.We called a webmethod asynchronously from a webpage.Then the page will be rendered from the server to the client,but the result of the called method may be reaches the webservice after rendering the site.So the webservice become not able to send the result to webform.My question is how to inform the webform asynchronously without polling for its completion and too much delay?
Plz Help................
|
|
|
|
|
The whole point of asynchronous calls is that you get notified when they're done. If you're gonna return before it's done, then you've left yourself no choice but to poll in some fashion.
I suggest you go for a long walk, and ponder the motivations behind building asynchronous HTTP request support into browsers...
But who is the king of all of these folks?
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
