Hi.. I will be posting all the testing related stuff here. The content posted here is a collection from different websites.

Thursday, June 14, 2012

LoadUI for Load Testing

LoadUI is a free load testing tool that is customized to make load testing in SoapUI easier and more efficient. LoadUI allows users to easily distribute the load Tests to any desired number of LoadUI Agents by simply dragging and dropping the tests as required which makes it easy to simulate load on the application. There are various components of this tool. Below is a brief overview of the components available.

1. Generators:

The generators available in LoadUI are the components which generate a load of samples or send a trigger message at a certain rate over a period of time. Different types of generators are available like Fixed Load, fixed rate, ramp, random, variance, virtual user generators. All generators have an On/Off button on their top-left, allowing you turn them off while the Load Test is running.

2. Runners:

Runners are the components actually performing some kind of action against your target system and outputting the results, for example sending a Web Page request or executing a SoapUI TestCase. To get you started there are few runners available like web page runner, Soap UO runner, script runner, process runner etc.
Runners will handle any incoming message as a trigger and use any parameters in the incoming message as applicable. They will have at least two output terminals, one for publishing the results for each request and one for publishing the current number of running requests (a new message is sent every time this number changes).

3. Analysis:

Analysis components are used for analysing and asserting results output from a runner. There are two analysis components currently available:
These are
  • Statistics - calculates and outputs statistics on results from a runner. The graph is updated in real-time as configured and its contents are continuously written to the output-terminal, which could be used for asserting, logging, etc.
  • Assertion - asserts incoming message values, for example output from a runner or a statistics component or response times from a runner component. You can of course add as many assertion components as required to assert any desired response times, etc., in your Load Test.

4. Flow

Flow components are used to control the flow of the Load Test execution, for example to divide load between different components, filter messages, etc. There are currently two router components available: Splitter and Delay.
  • Splitter - divides incoming messages evenly or randomly between the configured output terminals, allowing you to for example split the trigger messages from a Load Generator to go to different runners.

  • Delay - pauses the incoming messages for the configured time, for example you could put this between two web-page runners which would delay the sequential execution between them.

5. Scheduler:

Schedulers are of 2 types – Scheduler and Interval
  • Scheduler: Scheduler display in the above figure gives an overview of the configured schedule and shows the current position in the schedule.
  • Interval: Sends a start message after a determined interval (Start At), followed by a stop message after a certain period of time (Duration).

6. Output:

Output components are for output messages from any other component to some destination. Currently only one component is available for this:
The TableLog is useful for visualizing individual results, messages, etc. It has settings allowing you to write its contents to a properties file, making it a good option for logging responses, assertion failures, statistics, etc.

7. Misc

  • Note - Used to add notes to the workspace, which you can use to add text to the workspace
  • soapUI MockService - This will allow you to add a soapUI MockService to your project, in order to simulate a Web Service.

Cross-browser Testing

What is Cross-browser Testing

Cross-browser  refers to the ability for a website, web application, HTML construct or client-side script to function correctly across all, or the majority of web browsers. The term cross-browser is often confused with multi-browser. Multi-browser is a new paradigm in web development that allows a website or web application to provide more functionality over several web browsers, while ensuring that the website or web application is accessible to the largest possible audience without any loss in performance. Cross-browser capability allows a website or web application to be properly rendered by all browsers.

Need of CBT -

The term cross-browser has existed since web development began.

1- The browser market has broadened, and to claim cross-browser compatibility, the website is nowadays expected to support browsers such as Mozilla Firefox, Opera, Google Chrome and Safari in addition to Internet Explorer and Netscape.

2- There has been an attitude shift towards more compatibility in general. Thus, some degree of cross-browser support is expected and only its absence needs to be noted.

Challenges –

Making a cross-browser site is usually pretty simple for basic websites. However, complex sites with a lot of HTML formatting and JavaScript may require significant extra coding in order to be compatible with multiple browsers. Some developers may even generate completely different pages for each browser. While CSS formatting has helped standardize the appearance of Web pages across multiple browsers, there are still several inconsistencies between Web browsers. Therefore, cross-browser design continues to be a necessary aspect of Web development. Developing professional websites can be a daunting task, especially when your client wants all the bells and whistles. After adding more and more of these fancy effects and complex layouts, the chances that your website might not display 100% correctly in other browsers rises. Microsoft now believes in web-standards (finally!!) and this could definitely make our lives as web devs/designers easier. Their new product line, Expression, aims to compete with Adobe Flash in the rich media internet experience, and they are claiming that Expression will build standards-based websites out of the box.

TIPS while working on Cross Browser Development –

There will never be an escape from tweaking small things to look perfect in every browser, but there are some tips that can help you develop a better, more browser-friendly website. For this one need to obtain a copy of VMware Server which is completely free. After that, install an old copy of Windows XP on the VM, and load it with all of the different browsers you could find. it is now possible to install multiple versions of Internet Explorer, and have them run synchronous in standalone mode.

List of Multiple Browsers available in Market –

Microsoft Internet Explorer v3/v4/v5.01/v5.5/v6
Mozilla Firefox v1.5/v2.0.0.6
Opera v9.22
Safari v3.0 beta

CBT Automation Tools –

Selenium RC
QTP 11

Useful Links –

http://www.my-debugbar.com/wiki/IETester/HomePage
http://crossbrowsertesting.com/
http://www.browsrcamp.com/
http://litmus.com/
http://www.netrenderer.com/
http://browsershots.org/

How to Test Web Applications against SQL Injection Attacks


SQL injection is a technique often used to attack databases through a website.

Definition: A malicious user could provide unexpected inputs to the application that are then used to frame and execute SQL statements on the database.

Explained As:
Injection is done by including portions of SQL statements in a web form entry field in an attempt to get the website to pass a newly formed rogue SQL command to the database (e.g. dump the database contents to the attacker).

Using this an attacker can do-
  • Bypassing Logins
  • Accessing secret data
  • Modifying contents of website
  • Shutting down the My SQL server

Since the consequences of allowing the SQL injection technique could be severe, SQL injection should be tested during the security testing of an application.

Example: User need to enter username and Password

User enters “ John ” and “ Smith ” as details. So SQL will be formed as
SELECT * FROM Users WHERE UserName = ‘John’ AND Password = ‘Smith’;

But if user enters UserName like “John’- “. Then SQL be formed as
SELECT * FROM Users WHERE UserName = ‘John’– AND Password = ‘Smith’;

Note that the part of the SQL statement after John is turned into a comment. If there were any user with the user name of John in the Users table, the application could allow the tester to log in as the user John. The tester could now view the private information of the user John.

Technical Implementations
  • Incorrectly filtered escape characters
This form of SQL injection occurs when user input is not filtered for escape characters and is then passed into an SQL statement.

Users can type input in user name like - John’ OR ‘1’=’1’
SELECT * FROM users WHERE name = '' OR '1'='1';
This will fetch any member with John as username.
Or If he can enter in username like a';DROP TABLE users; SELECT * FROM userinfo WHERE 't' = 't

This will build SQL like
SELECT * FROM users WHERE name = 'a';DROP TABLE users; SELECT * FROM userinfo WHERE 't' = 't';

  • Incorrect type handling
This form of SQL injection occurs when a user-supplied field is not strongly typed or is not checked for type constraints. This could take place when a numeric field is to be used in a SQL statement, but the programmer makes no checks to validate that the user supplied input is numeric

Statement := "SELECT * FROM userinfo WHERE id = " + a_variable + ";"

So user need to enter a numeric value for the “id”. If he enters like “ 1;Drop table users “
SQL build as
SELECT * FROM userinfo WHERE id=1;DROP TABLE users;

  • Conditional responses
One type of blind SQL injection forces the database to evaluate a logical statement on an ordinary application screen.

SELECT booktitle FROM booklist WHERE bookId = 'OOk14cd' AND '1'='1';

This will fetch all the book from the table though user do not have any particular search criteria.

Mitigation

Need to consider the below thing while doing the coding
  • Parameterized statements
  • Escaping
  • Pattern check
  • Database Permissions

References

Security Testing for Web Application




What is Security Testing?
Security testing is a process to determine that the information system protects data and maintains functionality as intended.
Security testing can also be defined as the process that determines that confidential data stays confidential and users can perform only those tasks that they are authorized to perform.



Six concepts of Security Testing
  • Confidentiality

Security measure that protects the disclosure of data or information to parties other than the intended.
  • Integrity

Whether the intended receiver receives the information or data which is not altered in transmission.
  • Authentication

Allows a receiver to have confidence that information it receives originated from a specific known source.
  • Authorization

The process of determining that a requester is allowed to receive a service or perform an operation.
  • Availability

Assuring information and communications services will be ready for use when expected.
  • Non-repudiation

Interchange of authentication information with some form of provable time stamp e.g. with session id etc.



Some Security problems in Web Applications
  • SQL injection

In this vulnerability, SQL queries can be injected in the form of user input data which can results in number of insecure behavior. For example, on a login page if the application is not protected against SQL injection, then anyone can use it to get all the user names and passwords stored in the database. This technique is mostly used in situations where SQL query is dynamically generated using the data or parameters supplied by user. This vulnerability can be extremely dangerous since SQL is often used for authentication, authorization, billing etc.

  • Cross Site Scripting (XSS)

Attacker can use this method to execute malicious script or URL on victim’s browser. Using cross-site scripting, attacker can use scripts like JavaScript to steal user cookies and information stored in the cookies.
Many web applications get some user information and pass this information in some variables from different pages.
E.g.: http://www.examplesite.com/index.php?userid=123&query=xyz
Attacker can easily pass some malicious input or <script> as a ‘&query’ parameter which can explore important user/server data on browser.

  • Spoofing


The creation of hoax look-alike websites or emails is called spoofing.



Tools used for Security Testing
  • Netsparker


This tool can detect SQL Injection + cross-site scripting issues.
Once a scan is complete, it displays the solutions besides the issues and enables the user to see the browser view and HTTP request/response.

  • Websecurify

Websecurify is a very easy-to-use and open sourcetool which automatically identifies web application vulnerabilities by using advanced discovery and fuzzing technologies.
It can create simple reports (that can be exported into multiple formats) once ran.
The tool is also multilingual and extensible with the add-on support.

  • Wapiti

Wapiti is an open source and web-based tool that scans the web pages of the deployed web applications, looking for scripts and forms where it can inject data.
It is built with Python and can detect:
- File handling errors (Local and remote include/require, fopen, readfile…)
- Database, XSS, LDAP and CRLF injections (HTTP response splitting, session fixation)
- Command execution detection (eval(),system(), passtru())

  • N-Stalker

The free edition performs restricted-yet-still-powerful set of web security assessment checks compared to the paid versions of the application.
It can check up to 100 web pages at once including web server and cross-site scripting checks.

  • Scrawlr

Scrawlr is free software for scanning SQL injection vulnerabilities on the web applications.
It is developed by HP Web Security Research Group in coordination with Microsoft Security Response Center.

  • Watcher

It is a plug-in for Fiddler and works as a passive-analysis tool for HTTP-based web applications.
Watcher runs silently in the background and interacts with the web-application to apply 30+ tests (where new ones can be added) while the user browses.
It will identify issues like cross-domain form POSTs, dangerous context-switching between HTTP and HTTPS, etc.



  • X5s


X5sis again a plug-in for Fiddler just like Watcher which is designed to find encoding and character transformation issues that can lead to XSS vulnerability.
It simply tests user-controlled input using special characters like <, >, ', and reviews how the output encodes how the output encodes the special characters.

Saturday, June 14, 2008

1. If I created an application with .pcs extension how will u test in IE?
Ans Pcs is one of the picture storage file extensions like jpeg,bmp etc.So the file should be downloaded and displayed in the same fashion as a bmp or a jpeg file does.

2. How will you test login page? Can we apply sql injection there?
Ans Login page can be tested using the validations framework.Yes we can apply the sql injection if there are too many number of users and these have to be crosschecked with the database values.

3. How will you change java script in IE?
Ans Open java script file in IE ..then go to View - > source

4. Can IE 3.,IE 4,IE 5,IE 6 work simultaneously on same computer
Ans No it is not possible

5 If I wrote a program , that has to work for Korea language but it is showing English words in the application ?what might have gone wrong
Ans The language console has to be changed

6. How do an org know that their page is viewed?
Ans If it is asp.net -> In global.asa file we have to set application scope -> create and initialize the page hit counter object.

7.How to kill a process in windows with out using UI (i.e. Task Manager)?
Ans By using Kill utility.This is the command we have to use kill processid

8. How can you do that in LINX OS?
Ans Kill system call

9. How do you change the DATE of the computer system with out using UI (i.e. At command prompt)?
Ans At the time of booting-> select the setup option -> System settings

10. How do you test a Login page which contains a username and password fields and a submit button?
Ans The username & password can be tested using the validations framework.Submit button can be tested using the actions framework.

11. If you are a Test Manager, How do you limit the size of password and username fields?
Ans Using Boundary value analysis


12.How do you improve your computer performance while you are testing an application
Ans. Do not open multiple instances of the application

13 How is the security works between Browser and Webserver(Certificate)?
Ans Using SSL ( https)

14 Difference between boxing and un boxing?(.Net or Java 1.5)?
Ans Boxing means coverts primitive data values to Objects.
Unboxing is nothing but Object to primitive.

15 Command to find Linux version, IP Address?
Ans ifconfig

16. What's the length of IP Address?
Ans 32 bit

17 can u open msaccess file through excel
Ans Yes …


18 How do you track testing of applications?
Ans For tracking the testcases we generally use a QA workshop(website where the testcases and its status is updated)
For tracking the bugs we use a bug tracking tool

19 How do you ensure that testing is enough for an application?
Ans When the entry and exit criteria is fulfilled

20 How do you test the Windows Update process?
Ans By verifying the sizes and timestamps of the files which got updated.

21 How can you access/change username and password of windows users?
Ans . Right Click on My Computer ->Manage ->Local users and Groups ->Users

22 Difference between .EXE and .DLL?
Ans Exe is an executable file which can be used to run any application on windows platform.
DLL is a set of libraries which is used to support the exe file.


These answers I will send on Monday :-

1. What are test cases for yahoo messenger?
What is proxy



12. some brief testcases to test security?
13. How do you test Javascript?

14. List the testcases for a Login page ?

15. Write various testcases to test a NotePad and how do you perform Load test?

How do you test the desktop calculator

16. How do you test DeskTop applications?


17. How do test the Installations

18. How do test the Winamp software?

how do u debug windows application

19. write test cases for ‘Down loading and install real player software in local machine’?

20. write test cases for standalone system?

21. test cases for winword/paint
22. test cases for - how do u write test cases for installation of application
23. how do u test word app in browser
1. Tell me about your current project?
I have straight away started with my current project
Global OutLook:--
This is an Microsoft’s project which includes a bunch of administrators ,resellers ,Organizations and users .The core idea of this project is to promote an corporate mail i.e., suppose If we consider Our company Symbiosys technologies i.e. we to our employee id as emp@sys.com so ,we have to create and the whole burden is on the company for creating users.
So, if at all we got a site where already these burden was taken by the Global Outlook the only thing we have to do is just register our company and basing on our preferences we have to pay to GO.

2. Tell me about your roles in testing your project?
This project has four levels first the GO administrator, His responsibility is to create resellers, check out the various options created by users, and create other administrators.
The reseller role is to create organizations along with reseller administrators
The organizations can be created through reseller or by sign up page
And billing will be done only if he registers through sign up page. There he can choose between free 30 days trail period or the full package.
Finally the user’s login where he can only send mails and change his details.

My role is to test whether the preference selected by the organizations are assigned properly or not, check for various permissions for different role as mentioned earlier ,check whether the billing is done properly basing on the organization’s choice with respect to database.

So, finally he told that so this is an internet mailing system and I h ave answered exactly sir.

3. Tell me about your self?
I have graduated in computer science and engineering from Raghu Engineering college in the year 2005 and since the have been working with Symbiosys technologies as a test engineer.
The he enquired where about of my college.

4. What is a cookie?
I stared that cookies are small text files placed on our computer by the web server to uniquely identify the web browser or the computer to which the server had to return the web page.
This makes the server to stack the pages in future may be requested by the user
This saves the time for web page retrieval. And instead of requesting the users choice for every page these can be used for page validation.




5. What is java script?
o Java script is a scripting language used for client side validations.
o This is an interpreted language.
o This can be used for validation on forms.
o Check for the simple calculations.
o Can be use for blocking certain fields being entered by the user.
o Generally the scripts are embedded into html code.
o However, there are uses for javascript:
• Browser Detection
Detecting the browser used by a visitor at your page. Depending on the browser, another page specifically designed for that browser can then be loaded.
• Cookies
Storing information on the visitor's computer, then retrieving this information automatically next time the user visits your page. This technique is called "cookies".
• Control Browsers
Opening pages in customized windows, where you specify if the browser's buttons, menu line, status line or whatever should be present.
• Validate Forms
Validating inputs to fields before submitting a form.
An example would be validating the entered email address to see if it has an @ in it, since if not, it's not a valid address.

6. How can developer handle the situation if the script is disabled?
I told that these are the serious problems with java script and if it is disabled we cannot access the pages objects , He then asked me that have u tried these with g mail but I told that I have tried all these with my project and if the script is disabled we cannot access these object in the web page.
So, a solution may be the developer give some pop up message to user to enable java script.(This may not be the right solution so, please know it for ur self )

7. Where do you disable the script?
ToolsInternet options  security  custom level  change properties of java script disable them.



8. What are test cases for yahoo messenger?
I told that usability, compatibility, performance, password hacking in security, various resolutions, voice mail, web camera functionality etc.

9. What is sql injection?
This is process of getting required information through SQL command


10. What is registry?
It is a collection of users information, system configuration , database various software and hardware information.
Registry consists of hives --> keys sub keys values  binary ,word and string.
11. What is Linux scripting?
I told that i just know basic Linux internal commands and not much scripting as I was working on windows.
Q: What is UTF-16?
A: UTF-16 uses a single 16-bitcode unit to encode the most common 63K characters, and a pair of 16-bit code unites, called surrogates, to encode the 1M less commonly used characters in Unicode.
Originally, Unicode was designed as a pure 16-bit encoding, aimed at representing all modern scripts. (Ancient scripts were to be represented with private-use characters.) Over time, and especially after the addition of over 14,500 composite characters for compatibility with legacy sets, it became clear that 16-bits were not sufficient for the user community. Out of this arose UTF-16. [AF]
Q: What is the definition of UTF-8?
A. UTF-8 is the byte-oriented encoding form of Unicode. For details of its definition, see Section 2.5 “Encoding Forms” and Section 3.9 “ Unicode Encoding Forms ” in the Unicode Standard. See, in particular, Table 3-5 UTF-8 Bit Distribution and Table 3-6 Well-formed UTF-8 Byte Sequences, which give succinct summaries of the encoding form. Also see sample code which implements conversions between UTF-8 and other encoding forms. Make sure you refer to the latest version of the Unicode Standard, as the Unicode Technical Committee has tightened the definition of UTF-8 over time to more strictly enforce unique sequences and to prohibit encoding of certain invalid characters. There is an Internet RFC 3629 about UTF-8. UTF-8 is also defined in Annex D of ISO/IEC 10646. [MD Q: Can Unicode text be represented in more than one way?
A: Yes, there are several possible representations of Unicode data, including UTF-8, UTF-16 and UTF-32. In addition, there are compression transformations such as the one described in the Unicode Technical Report #6: A Standard Compression Scheme for Unicode. [MD]
Q: What is a UTF?
A: A Unicode transformation format (UTF) is an algorithmic mapping from every Unicode code point (except surrogate code points) to a unique byte sequence. The ISO/IEC 10646 standard uses the term “UCS transformation format” for UTF; the two terms are merely synonyms for the same concept.
Each UTF is reversible, thus every UTF supports lossless round tripping: mapping from any Unicode coded character sequence S to a sequence of bytes and back will produce S again. To ensure round tripping, a UTF mapping must also map all code points that are not valid Unicode characters to unique byte sequences. These invalid code points are the 66 noncharacters (including FFFE and FFFF), as well as unpaired surrogates.
The SCSU compression method, even though it is reversible, is not a UTF because the same string can map to very many different byte sequences, depending on the particular SCSU compressor

What is the difference between SSH and SSL?
________________________________________
SSH (Secure Shell) and SSL (Secure Sockets Layer) can both be used to secure communications across the Internet. This page tries to explain the differences between the two in easily understood terms.
SSL was designed to secure web sessions; it can do more, but that's the original intent.
SSH was designed to replace telnet and FTP; it can do more, but that's the original intent.
SSL is a drop-in with a number of uses. It front-ends HTTP to give you HTTPS. It can also do this for POP3, SMTP, IMAP, and just about any other well-behaved TCP application. It's real easy for most programmers who are creating network applications from scratch to just grab an SSL implementation and bundle it with their app to provide encryption when communicating across the network via TCP. Check out: stunnel.org.
SSH is a swiss-army-knife designed to do a lot of different things, most of which revolve around setting up a secure tunnel between hosts. Some implementations of SSH rely on SSL libraries - this is because SSH and SSL use many of the same encryption algorithms (i.e. TripleDES).
SSH is not based on SSL in the sense that HTTPS is based on SSL. SSH does much more than SSL, and they don't talk to each other - the two are different protocols, but have some overlap in how they accomplish similiar goals.
SSL by itself gives you nothing - just a handshake and encryption. You need an application to drive SSL to get real work done.
SSH by itself does a whole lot of useful stuff that allows users to perform real work. Two aspects of SSH are the console login (telnet replacement) and secure file transfers (ftp replacement), but you also get an ability to tunnel (secure) additional applications, enabling a user to run HTTP, FTP, POP3, and just about anything else THROUGH an SSH tunnel.
Without interesting traffic from an application, SSL does nothing. Without interesting traffic from an application, SSH brings up an encrypted tunnel between two hosts which allows you to get real work done through an interactive login shell, file transfers, etc.
Last comment: HTTPS does not extend SSL, it uses SSL to do HTTP securely. SSH does much more than SSL, and you can tunnel HTTPS through it! Just because both SSL and SSH can do TripleDES doesn't mean one is based on the other.


As a parent you may have concerns about the content your children encounter as they surf the Web. Internet Explorer 6 helps you safeguard your family's browsing experience with Content Advisor, which can be used to control the Web sites that your family can view. With Content Advisor, you can give your children access to a specific list of Web sites that you allow and prevent them from accessing others. Find out how to use it so you can rest easier.

Test suite
From Wikipedia, the free encyclopedia
(Redirected from Executable test suite)
Jump to: navigation, search
To meet Wikipedia's quality standards and make it more accessible to a general audience, this article may require cleanup.
The introduction to this article provides insufficient context for those unfamiliar with the subject matter.
Please help Wikipedia by improving the introduction according to the guidelines laid out at Wikipedia:Guide to layout. You can discuss the issue on the talk page.

The most common term for a collection of test cases is a test suite. The test suite often also contains more detailed instructions or goals for each collection of test cases. It definitely contains a section where the tester identifies the system configuration used during testing. A group of test cases may also contain prerequisite states or steps, and descriptions of the following tests.
Collections of test cases are sometimes incorrectly termed a test plan. They may also be called a test script, or even a test scenario.
An executable test suite is a test suite that is ready to be executed. This usually means that there exists a test harness that is integrated with the suite and such that the test suite and the test harness together can work on a sufficiently detailed level to correctly communicate with the system under test (SUT).
The counterpart of an executable test suite is an abstract test suite. However, often terms test suites and test plans are used, roughly with the same meaning as executable and abstract test suites, respectively.
[edit]
See also
test harness
In software testing, a test harness is a collection of software tools and test data configured to test a program unit by running it under varying conditions and monitor its behavior and outputs. It has two main parts namely, Test execution engine and the test script repository.
This entry is from Wikipedia, the leading user-contributed encyclopedia. It may not have been reviewed by professional editors (see full disclaimer)
localization
Customizing software and documentation for a particular country. It includes the translation of menus and messages into the native spoken language as well as changes in the user interface to accommodate different alphabets and culture. See internationalization and l10n
globalization
Operating around the world. Although many large companies have globalized for decades, the Web, more than any other phenomenon, has enabled the smallest company to have a global presence. See localization.
internationalization
The support for monetary values, time and date for countries around the world. It also embraces the use of native characters and symbols in the different alphabets. See localization, i18n, Unicode and IDN
Briefcase
In Windows 95/98, a system folder used for synchronizing files between two computers, typically a desktop and laptop computer. Files to be worked on are placed into a Briefcase, which is then transferred to the second machine via floppy, cable or network. The Briefcase is then brought back to the original machine after its contents have been edited on the second machine, and a special update function replaces the original files with the new ones.





DNS
(Domain Name System) A system for converting host names and domain names into IP addresses on the Internet or on local networks that use the TCP/IP protocol. For example, when a Web site address is given to the DNS either by typing a URL in a browser or behind the scenes from one application to another, DNS servers return the IP address of the server associated with that name.
In this hypothetical example, WWW.COMPANY.COM would be converted into the IP address 204.0.8.51. Without DNS, you would have to type the four numbers and dots into your browser to retrieve the Web site, which of course, you can do. Try finding the IP of a favorite Web site and type in the dotted number instead of the domain name!
Your continued donations keep Wikipedia running!
Web crawler
From Wikipedia, the free encyclopedia
Jump to: navigation, search

It has been suggested that this article or section be merged with Spidering. (Discuss)

See WebCrawler for the specific search engine of that name.
A web crawler (also known as a web spider or web robot) is a program which browses the World Wide Web in a methodical, automated manner. Other less frequently used names for web crawlers are ants, automatic indexers, bots, and worms (Kobayashi and Takeda, 2000).
Web crawlers are mainly used to create a copy of all the visited pages for later processing by a search engine, that will index the downloaded pages to provide fast searches. Crawlers can also be used for automating maintenance tasks on a web site, such as checking links or validating HTML code. Also, crawlers can be used to gather specific types of information from Web pages, such as harvesting e-mail addresses (usually for spam).
A web crawler is one type of bot, or software agent. In general, it starts with a list of URLs to visit, called the seeds. As the crawler visits these URLs, it identifies all the hyperlinks in the page and adds them to the list of URLs to visit, called the crawl frontier. URLs from the frontier are recursively visited according to a set of policies


Spidering
From Wikipedia, the free encyclopedia
Jump to: navigation, search

It has been suggested that this article or section be merged into Web Crawler. (Discuss)
"Spidering" is the process of using an automated script or bot to go to one or many websites and pull information to be stored for later use. The script can be targeted towards any set of information desired by the author.
Many legitimate sites use spidering as a means of providing up to date data. Froogle is a good example -- type a product into the froogle search and froogle will spider sites to retrieve the most current prices available.
As spiders can search data much quicker and in greater depth than human searches, they can have a crippling impact on the performance of a site. Needless to say if a single spider is performing multiple searches per second and demanding full result sets, a server would have a hard time keeping up with requests from multiple spiders.
"Spidering" is a synonym for "web crawling" (see "Web Crawler").

Search


Focus on Linux grep Command



Related Terms
• awk Command






Definition: Unix command "grep" allows you to search for a pattern in a list of files. Such patterns are specified as "regular expressions", which in their simplest form are "strings", such as words or sentence fragments.
The way we search for a string with grep is to put the words we are searching for together in single quotes.
• The syntax: % grep pattern file-name-1 file-name-2 …file-name-n
• An example: % grep 'mountain bike' sports hobbies
As a result of entering this command the operating system will print all the lines in the file "sports" and the file "hobbies" that contain the string "mountain bike". By default the line will be printed on the computer screen (in the shell window, where the command was issued).






How a Web Server Works
You can see from this description that a Web server can be a pretty simple piece of software. It takes the file name sent in with the GET command, retrieves that file and sends it down the wire to the browser. Even if you take into account all of the code to handle the ports and port connections, you could easily create a C program that implements a simple Web server in less that 500 lines of code. Obviously, a full-blown enterprise-level Web server is more involved, but the basics are very simple.
Most servers add some level of security to the serving process. For example, if you have ever gone to a Web page and had the browser pop up a dialog box asking for your name and password, you have encountered a password-protected page. The server lets the owner of the page maintain a list of names and passwords for those people who are allowed to access the page; the server lets only those people who know the proper password to see the page. More advanced servers add further security to allow an encrypted connection between server and browser, so that sensitive information like credit card numbers can be sent on the Internet.

How Search Engines Work
By Danny Sullivan, Editor-In-Chief
October 14, 2002
The term "search engine" is often used generically to describe both crawler-based search engines and human-powered directories. These two types of search engines gather their listings in radically different ways.
Crawler-Based Search Engines
Crawler-based search engines, such as Google, create their listings automatically. They "crawl" or "spider" the web, then people search through what they have found.
If you change your web pages, crawler-based search engines eventually find these changes, and that can affect how you are listed. Page titles, body copy and other elements all play a role.
Human-Powered Directories
A human-powered directory, such as the Open Directory, depends on humans for its listings. You submit a short description to the directory for your entire site, or editors write one for sites they review. A search looks for matches only in the descriptions submitted.
Changing your web pages has no effect on your listing. Things that are useful for improving a listing with a search engine have nothing to do with improving a listing in a directory. The only exception is that a good site, with good content, might be more likely to get reviewed for free than a poor site.
"Hybrid Search Engines" Or Mixed Results
In the web's early days, it used to be that a search engine either presented crawler-based results or human-powered listings. Today, it extremely common for both types of results to be presented. Usually, a hybrid search engine will favor one type of listings over another. For example, MSN Search is more likely to present human-powered listings from LookSmart. However, it does also present crawler-based results (as provided by Inktomi), especially for more obscure queries.

Win Registry
Starting with Windows 95, the Registry is a database that holds configuration data about the hardware and environment of the PC. It is made up of the SYSTEM.DAT and USER.DAT files.
The Registry can be edited directly, but that is usually only done for very technical enhancements or as a last resort. Routine access is done via the Windows control panels through the Properties option. Right clicking on almost every icon in Windows brings you the option of selecting Properties. See Win Properties.
Registry Details
To get into the Registry itself, run the Registry Editor program (REGEDIT.EXE) from the Run command in the Start menu. The Registry contains five folders. In Windows 95/98, there is a sixth folder.
Database normalization

In relational databases, normalization is a process that eliminates redundancy, organizes data efficiently, and reduces the potential for anomalies during data operations and improves data consistency. The formal classifications for quantifying "how normalized" a relational database are called normal forms (abbrev. NF).
A non-normalized database is vulnerable to data anomalies because it stores data redundantly. If data is stored in two locations, but later is updated in only one of the locations, then the data is inconsistent; this is referred to as an "update anomaly". A normalized database stores non-primary key data in only one location.
Normalized databases have a design that reflects the true dependencies between tracked quantities, allowing quick updates to data with little risk of introducing inconsistencies. Instead of attempting to lump all information into one table, data is spread out logically into many tables.

near pointer
In an x86 segmented address, a memory address within a single segment (the offset). Contrast with far pointer.

far pointer
In an Intel x86 segmented address, a memory address that includes both segment and offset. Contrast with near pointer.
DATABASE 2) A relational DBMS from IBM that was originally developed for its mainframes. It is a full-featured SQL language DBMS that has become IBM's major database product. Known for its industrial strength reliability, IBM has made DB/2 available for all of its own platforms, including OS/2, OS/400, AIX (RS/6000) and OS/390, as well as for Solaris on Sun systems and HP-UX on HP 9000 workstations and servers. See DB2 UDB.


Microsoft SQL Server
A relational DBMS from Microsoft that runs on Windows NT servers. It is Microsoft's high-end client/server database and a key component in its BackOffice suite of server products. SQL Server was originally developed by Sybase and also sold by Microsoft for OS/2 and NT. In 1992, Microsoft began development of its own version. Today, Microsoft SQL Server and Sybase SQL Server are independent products with some compatibility.
ipv4 is version 4 of the Internet Protocol (IP) and it is the first version of the Internet Protocol to be widely deployed. IPv4 is the dominant network layer protocol on the internet and when ignoring its successor — IPv6 — it is the only protocol used on the internet.
It is described in IETF RFC 791 (September 1981) which obsoleted RFC 760 (January 1980). The United States Department of Defense also standardized it as MIL-STD-1777.
IPv4 is a data-oriented protocol to be used on a packet switched internetwork (e.g., Ethernet). It is a best effort protocol in that it doesn't guarantee delivery. It doesn't make any guarantees on the correctness of the data; it may result in duplicated packets and/or packets out-of-order. All of these things are addressed by an upper layer protocol (e.g., TCP, UDP).
The entire purpose of IP is to provide unique global computer addressing to ensure that two computers over the internet can uniquely identify one another