|
All you Technically smart guys..
I have decide to work on a project of Search Engine on any one of 2 platform..JSP or PHP..
I have worked on various projects related to this theme earlier, but this time i want it to be working on Web.
The problem for me is "Crawl the Web"..
Could anyone please help that how can i Access the all websites URL..
OK let me explain:
In my previous projects i have already developed the concept of ranking a particular URL stored in my table according to frequency of keywords and little more concept..
So now i just wanted to ask two more things..
1)Could any one please tell me, will it be Possible for me to Crawl The Web i.e i want the all URL available on internet and not just few that are stored in my tables...
2)a little more concept on ranking of webpages like frquency...and now avoiding those fake keyword in webpages and various other good concepts that i can implement...
I will be very thankful for any kind of related help...
Please reply soon
|
|
|
|
|
You have no idea how over your head this subject is. It's a case of if you need to ask, you don't want to know...
May I suggest you read everything on the subject you can find; wikipedia has a lot of information.
Next you'll need to build a spider, it must be able to read eight pages at a time that's why we call them spiders, to go and read the web for you.
Buy some really big storage devices, you're going to need something in the region of 843 petabytes [1] to store the indexes alone.
The easy part is take the HTML from the spiders doing the crawling and get the keywords and any links.
How you rank the page is upto you, personally I would +1 for every reference to bunnies and -1 for cats.
[1] figures are +/- 10 orders of magnitude.
Panic, Chaos, Destruction.
My work here is done.
|
|
|
|
|
Well thanks for Information.
Please tell me isn't it possible to use some available spider code or i have to start from scratch.
And the thing that you mention about large storage, how can that be avoided by using Html from spider..as you mention..
please reply...
I just wanted to develop a working frame work..but not an actual Search Engine as compared to Google. [not even (20%) in comparison]
|
|
|
|
|
This reply shows your total lack of understanding. The spider [plenty of crawlers available - try searching] retrieves the data for you. You then need to index the data. Search engines DO NOT look at every web-site when performing a search, instead they use a keyword ind...
...why am I bothering? Really, why?
Panic, Chaos, Destruction.
My work here is done.
|
|
|
|
|
Thanks for understanding my lack of knowledge..
Respected sir..
I have 6 months to develop it and i just need maximum 2 weeks to understand the whole concept of CRAWLING.
Today here i just wanted to now -- What problem/consequences/difficulties will be faced by me when i will be having proper knowledge of this Concept almost like yours..
so just please tell me what concepts to LEARN and what will be difficult for me at that time...
Hope you get why you are Helping me??
Please reply..
|
|
|
|
|
Go back to my original answer. Look on wikipedia and READ the articles. There is a lot of information there so start reading.
- you need to be able to retrieve and index pages.
- as you index each page pull out the links.
- retrieve and index those.
The likes of google or bing are continuously crawling, looking for new or changed pages. The concept is simple, the practice is hard due to the sheer volume.
Start a crawler going on your / your college's home page and index ONLY that domain; see how big it gets.
Panic, Chaos, Destruction.
My work here is done.
|
|
|
|
|
Nagy Vilmos wrote: ...why am I bothering? Really, why?
Probably because you are a nice guy, who is motivated by a desire to help others. And every so often you get the sort of feedback that makes it all worthwhile. 
|
|
|
|
|
You should change that for a sarcasm icon!
I am only cuddly in the way that lions are cuddly. Hear me growl "gerr - row - well", scared now? 
Panic, Chaos, Destruction.
My work here is done.
|
|
|
|
|
AAArgh! Mummy! 
|
|
|
|
|
I am currently in a Distributed Systems Implementation class. My
group has created a project, which is a food ordering system. It is
coded in Java and uses Microsoft Access to make tables for the users,
orders, and items. In this class, we are required to connect our
program to two servers, one primary and another back-up. Our
professor expects the back-up server to be readily available on
another computer if the primary server fails on the first computer.
Can anyone explain to me how I would link my java coded proram with
access to two servers?
Thanks
|
|
|
|
|
try to connect to server1 - if it fails try server2.
|
|
|
|
|
How can i start windows calcultor from a java code...
|
|
|
|
|
Rather unclear here. Do you mean to start the windows calc program?
If yes, then have a look at launching external processes from java; check using this site[^].
Panic, Chaos, Destruction.
My work here is done.
|
|
|
|
|
hi everybody 
i'm trying to write a program that uses "brute force" to decrypt an encrypted message (using AES), where I only have the first 96-bits of the 128-bit secret key
when i complie i got this exception:
Exception in thread "main" javax.crypto.BadPaddingException: Given final block not properly padded
my code:
public static byte[] encrypt (byte[] plainText , SecretKeySpec spc)<br />
throws NoSuchAlgorithmException,GeneralSecurityException {<br />
Cipher cipher = Cipher.getInstance("AES");<br />
cipher.init(Cipher.ENCRYPT_MODE,spc);<br />
byte[] cipherText = cipher.doFinal( plainText);<br />
return cipherText ;<br />
}<br />
<br />
public static byte[] decrypt (byte[] cipherText , SecretKeySpec spec)<br />
throws NoSuchAlgorithmException,GeneralSecurityException {<br />
Cipher cipher = Cipher.getInstance("AES");<br />
cipher.init(Cipher.DECRYPT_MODE, spec);<br />
byte[] plainText = cipher.doFinal(cipherText);<br />
return plainText ;<br />
}<br />
<br />
public static void readfile(RandomAccessFile file, byte[] bytes) throws Exception<br />
{<br />
file.seek(0);<br />
int offset = 0;<br />
int numRead = 0;<br />
while (offset < bytes.length<br />
&& (numRead=file.read(bytes, offset, bytes.length-offset)) >= 0) {<br />
offset += numRead;<br />
}<br />
}<br />
<br />
public static void main(String[] args) throws Exception {<br />
try<br />
{<br />
RandomAccessFile cipherfile = new RandomAccessFile( "ciphertext2.dat", "r" );<br />
<br />
byte[] bytes = new byte[(int)cipherfile.length()];<br />
readfile(cipherfile,bytes);<br />
<br />
RandomAccessFile file = new RandomAccessFile( "partial-key.dat", "r" );<br />
byte[] keybytes = new byte[(int)file.length()+4];<br />
readfile(file,keybytes);<br />
for(int i=0;i<256;i++)<br />
{<br />
keybytes[12]=(byte)i;<br />
for(int j=0;j<256;j++)<br />
{<br />
keybytes[13]=(byte)j;<br />
for(int k=0;k<265;k++)<br />
{<br />
keybytes[14]=(byte)k;<br />
for(int l=0;l<265;l++)<br />
{<br />
keybytes[15]=(byte)l;<br />
SecretKeySpec key = new SecretKeySpec(keybytes, "AES");<br />
byte[] plain=new byte[(int)cipherfile.length()];<br />
plain=decrypt (bytes,key);<br />
for (int y=0;y<=plain.length;y++)<br />
System.out.print((char)plain[y]);<br />
}<br />
}<br />
}<br />
}<br />
<br />
file.close();<br />
<br />
cipherfile.close();<br />
}<br />
catch ( FileNotFoundException e)<br />
{<br />
System.out.println(e);<br />
}<br />
catch ( IOException e)<br />
{<br />
System.out.println(e);<br />
}<br />
}<br />
thanks in advance![Rose | [Rose]](https://codeproject.freetls.fastly.net/script/Forums/Images/rose.gif)
|
|
|
|
|
mesho wrote: for(int k=0;k<265;k++)
Are you sure this should be 265?
|
|
|
|
|
sorry, it was a typing error
what about my exception?any idea? 
|
|
|
|
|
mesho wrote: sorry, it was a typing error
Are you still getting the same error after correcting this? It looks like your encrypted data may be incorrect.
|
|
|
|
|
yes
i don't think that the problem is with encrypted data
how about reading encrypted data, and key?
is there any mistake?
|
|
|
|
|
mesho wrote: i don't think that the problem is with encrypted data
That is what the Padding Exception seems to be saying! You may need to look at the encryption side to see how the data looks at the end and then make certain that you reconstruct it in the same way during the decryption phase. Obviously if your key is wrong this may also affect the result.
|
|
|
|
|
Hey all,
let me start off by saying that I'm not totally sure of whether or not I'm phrasing my question right, but here goes. I'm trying to make a trading card game, and I want to make it expandable (not to mention I don't want to hard code the cards into the game). I was brainstorming the best way to have different cards (represented as objects) and let them have different actions in their methods (for example, onPlay() and onDiscard() will have different implementations for different cards). What I want to be able to do is, on initiation, have the program load these cards into an array of AbstractCard (which all card objects will implement), from which the game can use as its database of cards. At first, I thought of storing the actions in a text file, and then having the card determine its action based on what was saved in the text file. However, I quickly realized how messy that would be, and had a different idea. Perhaps I could create a different class for each card, and compile them all separately and put them in a location on the hard drive. The program would then check that directory and load all the compiled classes it finds there, declare objects from them, and put them in the AbstractCard array (to me, this sounds similar to what a plug-in system does).
So, my question(s) is(are), is this a good way of going about this? If so, how should I implement it? With regards to the implementation, this sounds kind of like reflection, but I have no experience in that area and so am not sure how to go about doing that.
Thanks,
Chuck Murphy
|
|
|
|
|
Inheritance is your friend! This is the classic example of using it.
Start with your nice basic abstract class:
abstract class Card
{
private String description;
Card(String description)
{
this.description = description;
}
abstract void onPlay();
abstract void onDiscard();
}
Then for each type of card you need, have another class:
class RealCard extend Card
{
RealCard()
{
super("This is a real card");
}
void onPlay()
{
}
void onDiscard()
{
}
}
That is the better way to design things.
The second part of how to store the objects is covered by a simple enough idea - serialization. Get your card classes right and hard code them to start with, then worry about the persistaance; it should be relatively easy.
Panic, Chaos, Destruction.
My work here is done.
|
|
|
|
|
Well, you see, I'm already doing that, I'm just looking for how to offer users the ability to create their own cards and add them to the game, without needing to get the source, add some classes, and recompile the game. Also, assume here that these users are trusted, I know someone could stick some malicious code in one of these user-created cards if I follow this method 
Here's some psuedo-code:
LinkedList<AbstractCard> lstAllCards = new LinkedList<AbstractCard>();
for each file in directory "user_cards/"
{
if file is a java class
{
AbstractCard newCard = file.instantiate()
lstAllCards.add(newCard);
}
}
From what I've read of reflection, the way to do this might be to try this:
AbstractCard newCard = Class.forName(fileName).newInstance();
Also, after reading this, I may want to have the card list be a LinkedList<Class>, not a LinkedList<AbstractCard>, so that I can instantiate new cards at will without having to refer back to these initial instances. Am I on the right track?
EDIT: angle brackets weren't showing up for the generics, replaced them with "& lt;" and "& gt";
|
|
|
|
|
I have searched high and low (and yes - even in the jCIFS library) on how to share a directory/folder in Java. I have seen it done in C# (the language I am more familiar with), but have not been able to figure out how to do it in Java. Please, Java experts please help me out and point me in the right direction!
|
|
|
|
|
I don't think you can because this is OS-specific behaviour. You could use JNI to call a Windows API or shell out to run NET SHARE, but I as far as I am aware you can't do it directly from within Java.
|
|
|
|
|
Oh no! Thank you so much for replying. The project I am working on has to work on multiple platforms (Window, Mac, Linux, Unix, etc).
I guess we might have to do it differently for each platform we connect to - I forgot to mention that I wanted to do it remotely. I am able to create a directory using the jCIFS library.
Do you know of a way to remotely determine what OS is running on machine we connect to - that is another issue I have yet to determine if is possible.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
