In a current scenario companies are trying to adopt big data analytics software to
anatomize the voluminous amount of data in real time. If we think about the size and intricacy
the databases used in
today’s enterprises, it is
a considerable oppugn for an enterprises to build their applications
that can be capable of handling voluminous amounts of
data. If we compare the heritage relational databases and non-relational NoSQL databases, the non-relational NoSQL databases
dynamic provisioning, significant
performance, horizontal scaling,
distributed architecture and developer agility benefits. Now a full-fledged NoSQL
databases raped up furthermore
event of reading is more instead of writing on voluminous data,
numerous applications that will shift to NoSQL
and will select it as
data storage system. Plenty of existing systems develop their service
to support the volatile ramp of data. We have proposed model to aid composite database structure inclusive
of a relational database (RDB) and NoSQL database. Model
tolerate error coming out
of application and concern with database conversion parallelly .we also
synchronization between both databases. A
is also provided to use RDB
NoSQL database parallelly.
migration, relational database to
database, document-oriented database
As cloud services have been grown, the big and composite database system are getting magnetize
now a days. In
NoSQL databases are
surging. Nearly all of present systems are based on relational database.
Companies have to manage big data with the aid of NoSQL database for study or accessing data in a short time period, as size of data increases. Instead of substituting relational database
NoSQL database, enterprise and research organizations integrate twain databases
.To handle meddle scale of data in user application, it will directly
interact with relational database. And the NoSQL play an important role as system
back-end data pool for analysis and R/W operations, or backup
after some specific time period.
this two databases may lead to damage the system design. In the real system, application interface by relational database with the help of SQL. At same period of time to use both the databases the application design need to be changed. At the time of integrating
real system to NoSQL database, a mechanism is to be needed for data
migration from relational to NoSQL.
Relational database is the most widely used data storage technology for many years. In today’s world requirement for data processing have prompted emergence of new data storage, retrieval and processing mechanism. One of such mechanism is NoSQL
called non-relational databases.
NoSQL databases is one of the mechanisms, and is called as non-relational
databases. Numerous companies sift to NoSQL
and managing data and
relational database system must be migrated to NoSQL. Due to
difference in the schema of NoSQL database and relational
have to learn a
new database. As the join operation
is not supported in the NoSQL, user have to collect
different table and leads to a poor performance. So the
conversion is must
migration to relational to NoSQL.
Designed NoSQL is termed as ‘Not Only SQL’ for distributed data store. To overcome the limitations of RDBMSs in storing and processing cloud big data, NoSQL databases
have been developed for large scale and high
applications. Web 2.0 applications needs large data storage with
attributes such as picture, video,
text, comments, and other information,
NoSQL databases are mainly designed to comply with the requirements of Web 2.0
applications. Key features of NoSQL
Ability to scale horizontally. 2. Ability to partition or distribute over many functions 3. Comparably weaker concurrency
than ACID. 4. Compared to SQL binding, simple call level protocol.
5. Ability to attach new
feature to data
records dynamically. 6. Capable use of RAM and distributed indexes for data storage.
Horizontal scaling, replication and distribution of data over
various purpose make data reading and writing operations
faster. ACID properties which are essential for data
consistency for the parallel transaction, but NoSQL does not
offer the ACID properties. In the distributed environment
main requirement is system scalability. The web based
applications mainly run
for a distributed system
it is not possible to ensure simultaneous Consistency, Availability and Partition tolerance at the same which is stated as the CAP theorem that articulates two of them can be achieved. A weaker model BASE (Basically
Available, Soft state, Eventual consistency)
replaces ACID in
order to keep NoSQL data consistent and reliable. Invented by Eric Brewer and according to BASE properties
described as: Basic availability: Any request will be responded with successful
or failed execution. Soft state: The state of the
system is ‘soft’ which may change over time. So due to eventual consistency
changes even may be going on without any input. Eventual consistency: Eventually the database will be
consistent even though it could be inconsistent momentarily.
Five categories of NoSQL databases are: column-oriented,
Document –oriented, key-value, graph, multi-model. The most
important types of databases for some years is key-value
store and document
databases. In proposed model we concentrate
NoSQL database as document-oriented database.
1. Key-Value -Like Riak, a Key Value databases
of unframed allotted dictionaries and schema less databases. They
do not have a default
schema. . The key
be simulated or autogenic, and the value can be anything
string, JSON, BLOB and rest.
2. Document – Most popular cuchbase and mongodb are the
types of document
database. As they do not have the default schema they are very adjustable in the content type.
They cooperate with distinct data types such as JSON, BSON, XML
BLOBs. Fundamentally they shows only a activity of key-
Using a key document is written/read. Apart
from utility key
value, a document databases add
discrete utilities to search the document on their content.
3. Column – Hbase and Hypertable are from big table type, they
category having default schema. After storing the data in a cell, they are groped in a columns and then columns are logically
join and form the families of columns. Apparently, they can accommodate
limitless number of columns (limited according to implementation) which possibly
made at operating time or at schema definition.
4. Graph-Oriented – This scheme be up to support complex data
errors which are also carry out in an approximately short period of time compared to rest databases
utilizing the schemes
databases give high
adjustability for any changes of an attribute by the database because they do not have a
predefined database schema.
II. COMPARATIV E ANALYSIS ON RDBMS VS.NOSQL.
points have been summarized from 8 as a part of
providing a comparative analysis on Relational databases and NoSQL data bases:
reliability: RDBMS support ACID properties to provide transaction
reliability whereas NoSQL databases are not reliable
RDBMSs because of its weaker BASE properties compared to ACID.
Data Model: Relational Databases are
on relational model where
that contain set of
the relation. On the other hand NoSQL databases take many modelling techniques
key value stores, document data store, column data store and graph data model
Scalability: The internet based web applications require
horizontal scalability as it spread over several servers in a distributed environment.
NoSQL data store support horizontal scalability whereas it is a great challenge for the relational model.
The relational databases
cannot handle schema less
unstructured data as it can work only with well-defined schema. But it is one of the requirements for handling cloud databases.
fit for the cloud scale solution as it fulfills all of the characteristics
which are desirable
Big data handling: Because of their issues with scalability and data distribution
it is not an easy task for relational database to handle big data. On the other hand
NoSQL databases designed to handle the big data distributed in the clustered environment.
Complexity: Day by day complexity
in relational databases
rises because of the continuous
changed requirements. If the data for the changed requirements does not fit in the
RDBMS schema, then it would make a complex
situation in terms of changing schema and related programming code. On the other hand there is no significant effect on NoSQL databases as they can store unstructured, semi-structured or structured data.
Crash Recovery: Recovery manager ensures crash recovery
for RDBMS data. On the other hand crash recovery depends on data replication for NoSQL databases. MongoDB uses Journal file as recovery mechanism.
Security: Very secure mechanisms are adopted by RDBMSs to secure their data. NoSQL databases are designed for storing and
handling big data, and subsequently providing higher performance at the cost of security. Security of information is a
concern of the newly evolving cloud
environment which is being considered as the next generation architecture for enterprises.
In this paper we focus on NoSQL database, namely
MngoDB.This paper proposed a framework that
integrates relational database and NoSQL database and handle database
migration. The main fetchers of the framework is as follows.
1. SQL Interface to relational and NoSQL Database. We offer SQL interface to access both relational
NoSQL database. With the use of this interface application does not
need to modify queries or handle NoSQL queries. So actual system design can access both
2. Database Converter. Database migration from relational to
database can be done. Features of
continuous synchronization also there.
III. DATA MIGRATION FRAMEWORK
The data migration framework is incredibly modularized, layered between application and databases. It is responsible for performing queries from applications
data migration between databases at the same time. The framework provides a SQL interface parsing query statements
to access both a relational database and a
We offer a mechanism to control the database migration
process and let applications perform queries whether or not
target data (table) are being migrated. After data migration done, we
provide a patch mechanism to
tables. We present the data migration framework with its design
implementation in following sections.
2.1 Actual system with Relational database only
2.2 System architecture with data migration framework and its
Mediator: Mediator is an interface which accepts queries from
applications, parses queries, extracts and sends necessary
information to controller. Parser can tell the difference between read and write queries and pass the information to controller to put write queries, which might be affected by transformation
progresses, in a
queue if necessary.
Mapping: Hear one
SQL queries translator is
translate the SQL query into no SQL query. If user wants to fetch the data it directly fetch from NoSQL server.
Meta data store: It store write queries data of appliction Migration tool: extract the data from SQL server transform that data into NoSQL format and then load data into the NoSQL
information systems is of the
assets for many of the businesses. Pushed by customer demands
and force by changes in technologies, companies from time to time migrate from
one information system to a different. Hence, knowledge from the bequest system should be migrated to the new system.
Data migration has two main steps: first, restructuring
of source data according to requirements of the new system, and
from the source to the new database.
The academic literature offers many approaches /methods for dealing with these steps: schema conversion,
approach, ETL (Extract,
Transform, and Load), program
migration, automated data migration.
We use ETL ((Extract, Transform, and Load)
for migration relational database to document oriented database. hear flow chart of data migration.
2.3 algorithm for migration
The execution process of migration of Relational to document
oriented database are as follows.
1. Get list of table from Relational database.
2. Get all the fields of table.
3. Fetch all values from relational databases with PSQL.
4. Format the
data fetch from
make compatible with NoSQL databases.
5. Check all the type of values is it supported to targeted NoSQL
6. If yes then insert data directly
into targeted database. Or do type cast make them suitable to targeted databases.