Contact at or 8097636691
Responsive Ads Here

Thursday, 8 February 2018

Smart Crawler: A Two-stage Crawler for Efficiently Harvesting Deep-Web Interfaces(2015)

Smart Crawler: A Two-stage Crawler for

 Efficiently Harvesting Deep-Web 


SOFTWARE MODEL OR ARCHITECTURE ANALYSIS: Structured project management techniques (such as an SDLC) enhance management’s control over projects by dividing complex tasks into manageable sections. A software life cycle model is either a descriptive or prescriptive characterization of how software is or should be developed. But none of the SDLC models discuss the key issues like Change management, Incident management and Release management processes within the SDLC process, but, it is addressed in the overall project management. In the proposed hypothetical model, the concept of user-developer interaction in the conventional SDLC model has been converted into a three dimensional model which comprises of the user, owner and the developer. In the proposed hypothetical model, the concept of user-developer interaction in the conventional SDLC model has been converted into a three dimensional model which comprises of the user, owner and the developer. The ?one size fits all? approach to applying SDLC methodologies is no longer appropriate. We have made an attempt to address the above mentioned defects by using a new hypothetical model for SDLC described elsewhere. The drawback of addressing these management processes under the overall project management is missing of key technical issues pertaining to software development process that is, these issues are talked in the project management at the surface level but not at the ground level.
                          A software cycle deals with various parts and phases from planning to testing and deploying software. All these activities are carried out in different ways, as per the needs. Each way is known as a Software Development Lifecycle Model (SDLC). A software life cycle model is either a descriptive or prescriptive characterization of how software is or should be developed. A descriptive model describes the history of how a particular software system was developed. Descriptive models may be used as the basis for understanding and improving software development processes or for building empirically grounded prescriptive models.
SDLC models The Linear model (Waterfall) - Separate and distinct phases of specification and development. - All activities in linear fashion. - Next phase starts only when first one is complete. * Evolutionary development - Specification and development are interleaved (Spiral, incremental, prototype based, Rapid Application development). - Incremental Model (Waterfall in iteration), RAD(Rapid Application Development) Focus is on developing quality product in less time, - Spiral Model - We start from smaller module and keeps on building it like a spiral. It is also called Component based development. * Formal systems development - A mathematical system model is formally transformed to an implementation. * Agile Methods. - Inducing flexibility into development. * Reuse-based development - The system is assembled from existing components.
The General Model
Software life cycle models describe phases of the software cycle and the order in which those phases are executed. There are tons of models, and many companies adopt their own, but all have very similar patterns.  Each phase produces deliverables required by the next phase in the life cycle. Requirements are translated into design. Code is produced during implementation that is driven by the design. Testing verifies the deliverable of the implementation phase against requirements.
SDLC Methodology:
Spiral Model
            The spiral model is similar to the incremental model, with more emphases placed on risk analysis.  The spiral model has four phases: Planning, Risk Analysis, Engineering and Evaluation.  A\ software project repeatedly passes through these phases in iterations (called Spirals in this model).  The baseline spiral, starting in the planning phase, requirements is gathered and risk is assessed.  Each subsequent spirals builds on the baseline spiral. Requirements are gathered during the planning phase.  In the risk analysis phase, a process is undertaken to identify risk and alternate solutions.  A prototype is produced at the end of the
risk analysis phase. Software is produced in the engineering phase, along with testing at
the end of the phase.  The evaluation phase allows the customer to evaluate the output of the project to date before the project continues to the next spiral. In the spiral model, the angular component represents progress, and the radius of the spiral represents cost. Spiral Life Cycle Model. 
           This document play a vital role in the development of life cycle (SDLC) as it describes the complete requirement of the system.  It means for use by developers and will be the basic during testing phase.  Any changes made to the requirements in the future will have to go through formal change approval process.
         SPIRAL MODEL was defined by Barry Boehm in his 1988 article, “A spiral Model of Software Development and Enhancement.  This model was not the first model to discuss iterative development, but it was the first model to explain why the iteration models.
        As originally envisioned, the iterations were typically 6 months to 2 years long.  Each phase starts with a design goal and ends with a client reviewing the progress thus far.   Analysis and engineering efforts are applied at each phase of the project, with an eye toward the end goal of the project.
The steps for Spiral Model can be generalized as follows:
·         The new system requirements are defined in as much details as possible.  This usually involves interviewing a number of users representing all the external or internal users and other aspects of the existing system.
·         A preliminary design is created for the new system.
·         A first prototype of the new system is constructed from the preliminary design.  This is usually a scaled-down system, and represents an approximation of the characteristics of the final product.
·         A second prototype is evolved by a fourfold procedure:
1.      Evaluating the first prototype in terms of its strengths, weakness, and risks.
2.      Defining the requirements of the second prototype.
3.      Planning an designing the second prototype.
4.      Constructing and testing the second prototype.
·         At the customer option, the entire project can be aborted if the risk is deemed too great.  Risk factors might involved development cost overruns, operating-cost miscalculation, or any other factor that could, in the customer’s judgment, result in a less-than-satisfactory final product.
·         The existing prototype is evaluated in the same manner as was the previous prototype, and if necessary, another prototype is developed from it according to the fourfold procedure outlined above.
·         The preceding steps are iterated until the customer is satisfied that the refined prototype represents the final product desired.
·         The final system is constructed, based on the refined prototype.
·         The final system is thoroughly evaluated and tested.   Routine maintenance is carried on a continuing basis to prevent large scale failures and to minimize down time.
  • High amount of risk analysis
  • Good for large and mission-critical projects.
  • Software is produced early in the software life cycle.
 Feasibility Study       
      Preliminary investigation examine project feasibility, the likelihood the system will be useful to the organization. The main objective of the feasibility study is to test the Technical, Operational and Economical feasibility for adding new modules and debugging old running system. All system is feasible if they are unlimited resources and infinite time. There are aspects in the feasibility study portion of the preliminary investigation:
·         Technical Feasibility
·         Operational Feasibility
·         Economical Feasibility                                                                             
   3.5.1 Economic Feasibility         
A system can be developed technically and that will be used if installed must still be a good investment for the organization. In the economical feasibility, the development cost in creating the system is evaluated against the ultimate benefit derived from the new systems. Financial benefits must equal or exceed the costs.
The system is economically feasible. It does not require any addition hardware or software. Since the interface for this system is developed using the existing resources and technologies available at NIC, There is nominal expenditure and economical feasibility for certain.                         
   3.5.2 Operational Feasibility  
Proposed projects are beneficial only if they can be turned out into information system. That will meet the organization’s operating requirements. Operational feasibility aspects of the project are to be taken as an important part of the project implementation. Some of the important issues raised are to test the operational feasibility of a project includes the following: -
·         Is there sufficient support for the management from the users?
·         Will the system be used and work properly if it is being developed and implemented?
·         Will there be any resistance from the user that will undermine the possible application benefits?
This system is targeted to be in accordance with the above-mentioned issues. Beforehand, the management issues and user requirements have been taken into consideration. So there is no question of resistance from the users that can undermine the possible application benefits.
The well-planned design would ensure the optimal utilization of the computer resources and would help in the improvement of performance status.                                                 
   3.5.3 Technical Feasibility       
The technical issue usually raised during the feasibility stage of the investigation includes the following:
·         Does the necessary technology exist to do what is suggested?
·         Do the proposed equipments have the technical capacity to hold the data required to use the new system?
·         Will the proposed system provide adequate response to inquiries, regardless of the number or location of users?
·         Can the system be upgraded if developed?
·         Are there technical guarantees of accuracy, reliability, ease of access and data security?
Earlier no system existed to cater to the needs of ‘Secure Infrastructure Implementation System’. The current system developed is technically feasible. It is a web based user interface for audit workflow at NIC-CSD. Thus it provides an easy access to the users. The database’s purpose is to create, establish and maintain a workflow among various entities in order to facilitate all concerned users in their various capacities or roles. Permission to the users would be granted based on the roles specified.  Therefore, it provides the technical guarantee of accuracy, reliability and security. The software and hard requirements for the development of this project are not many and are already available in-house at NIC or are available as free as open source. The work for the project is done with the current equipment and existing software technology. Necessary bandwidth exists for providing a fast feedback to the users irrespective of the number of users using the system.
4.3 FUNCTIONAL Requirements
Create  Site Database
Site Frontier
Site Exploring
Site Locating
Smart Crawler
4.4   NON Functional Requirements         
The major non-functional Requirements of the system are as follows
The system is designed with completely automated process hence there is no or less user intervention.
The system is more reliable because of the qualities that are inherited from the chosen platform java. The code built by using java is more reliable.
This system is developing in the high level languages and using  the advanced front-end and back-end technologies it will give response to the end user on client system with in very less time.
The system is designed to be the cross platform supportable. The system is supported on a wide range of hardware and any software platform, which is having JVM, built into the system.
The system is implemented in web environment using struts framework. The apache tomcat is used as the web server and windows xp professional is used as the platform.
Interface the user interface is based on Struts provides HTML Tag
Software Requirements:
Language                                    :       JDK (1.7.0)
Frontend                                     :       JSP, Servlets
Backend                                      :       Oracle10g
IDE                                             :       my eclipse 8.6
Operating System                       :      windows XP
Server                                          :        tomcat
Hardware Requirements:
Processor                                     :       Pentium IV
Hard Disk                                    :       80GB
RAM                                            :       2GB

No comments:

Post a Comment