In a current scenario companies are trying to adopt big data analytics software to
anatomize the voluminous amount of data in real time. If we think about the size and intricacy
of
the databases used in
today’s enterprises, it is
a considerable oppugn for an enterprises to build their applications
that can be capable of handling voluminous amounts of
data. If we compare the heritage relational databases and non-relational NoSQL databases, the non-relational NoSQL databases
are
better for
dynamic provisioning, significant
performance, horizontal scaling,
and
distributed architecture and developer agility benefits. Now a full-fledged NoSQL
databases raped up furthermore
the
event of reading is more instead of writing on voluminous data,
numerous applications that will shift to NoSQL
and will select it as
data storage system. Plenty of existing systems develop their service
to support the volatile ramp of data. We have proposed model to aid composite database structure inclusive
of a relational database (RDB) and NoSQL database. Model
can
tolerate error coming out
of application and concern with database conversion parallelly .we also
give
data
synchronization between both databases. A
mechanism
is also provided to use RDB
and
NoSQL database parallelly.

 

Keywords:
 database
 migration,  relational  database  to
 NoSQL

We Will Write a Custom Essay Specifically
For You For Only $13.90/page!


order now

database, document-oriented database

 

 

I.

INTRODUCTION

As cloud services have been grown, the big and composite database system are getting magnetize
now a days.   In
popularity of
 big  data
 NoSQL databases are
 also
 surging. Nearly all of present systems are based on relational database.
Companies have to manage big data with the aid of NoSQL database for study or accessing data in a short time period, as size of data increases. Instead of substituting relational database
by
NoSQL database, enterprise and research organizations integrate twain databases
.To handle meddle scale of data in user application, it will directly
interact with relational database. And the NoSQL play an important role as system
back-end data pool for analysis and R/W operations, or backup
after some specific time period.

 

The migration
of
this two databases may lead to damage the system design.  In the real system, application interface by relational database with the help of SQL. At same period of time to use both the databases the application design need to be changed. At the time of integrating
real system to NoSQL database, a mechanism is to be needed for data
migration from relational to NoSQL.

 

Relational database is the most widely used data storage technology for many years. In today’s world requirement for data processing have prompted emergence of new data storage, retrieval and processing mechanism. One of such mechanism is NoSQL
 databases  is
 also
 called  non-relational  databases.

NoSQL databases is one of the mechanisms, and is called as non-relational
databases. Numerous companies sift to NoSQL
database  and
 utilize
 it
 for
 storing 
and  managing data  and
relational database system must be migrated to NoSQL. Due to
the
difference in the schema of NoSQL database and relational
databases, user 
have to  learn a
 new database. As  the  join operation
is not supported in the NoSQL, user have to collect
the
data from
different table and leads to a poor performance. So  the
 schema
 conversion is  must
 for
 the    
 migration  to relational to NoSQL.

 

Designed NoSQL is termed as ‘Not Only SQL’ for distributed data store. To overcome the limitations of RDBMSs in storing and processing cloud big data, NoSQL databases
have been developed for large scale and high 
concurrency
applications. Web 2.0 applications needs large data storage with
flexible  schema
 for
 attributes such  as  picture,  video,
 text, comments, and other information,
the
NoSQL databases are mainly designed to comply with the requirements of Web 2.0
applications. Key features of NoSQL
databases
includes: 1.
Ability to scale horizontally. 2. Ability to partition or distribute over many functions 3. Comparably  weaker concurrency
than ACID. 4. Compared to SQL binding, simple call level protocol.

5. Ability to attach new
feature to data
records dynamically. 6. Capable use of RAM and distributed indexes for data storage.

 

Horizontal scaling, replication and distribution of data over
various purpose make data reading and writing operations
becomes
faster. ACID properties which are essential for data
consistency for the parallel transaction, but NoSQL does not
offer the ACID properties. In the distributed environment
the
main requirement is system scalability. The web based
applications  mainly  run
 on  such
 distributed
 environment.
And
for a distributed system
it is not possible to ensure simultaneous Consistency, Availability and Partition tolerance at the same which is stated as the CAP theorem that articulates two of them can be achieved. A weaker model BASE (Basically
Available, Soft state, Eventual consistency)
replaces ACID in
order to keep NoSQL data consistent and reliable. Invented by Eric Brewer and according to BASE properties
are
described as: Basic availability: Any request will be responded with successful
or failed execution. Soft state: The state of the
system is ‘soft’ which may change over time. So due to eventual consistency
changes even may be going on without any input. Eventual consistency: Eventually the database will be
consistent even though it could be inconsistent momentarily.

 

Five categories of NoSQL databases are: column-oriented,
Document –oriented, key-value, graph, multi-model. The most
important types of databases for some years is key-value
store and document
databases. In proposed model we concentrate
NoSQL database as document-oriented database.

 

1. Key-Value -Like Riak, a Key Value databases
are
of unframed allotted dictionaries and schema less databases. They

do not have a default
architecture
or
schema. . The key
can
be simulated or autogenic, and the value can be anything
like
string, JSON, BLOB and rest.

2. Document – Most popular cuchbase and mongodb are the

types of document
database. As they do not have the default schema they are very adjustable in the content type.  
They cooperate with distinct data types such as JSON, BSON, XML
and
BLOBs. Fundamentally they shows only a activity of key-
value databases.
Using a key document is written/read.  Apart
from  utility key
 value,  a  document databases add
 discrete utilities to search the document on their content.

3. Column – Hbase and Hypertable are from big table type, they
are
of columnar
category having default schema. After storing the data in a cell, they are groped in a columns and then columns are logically
join and form the families of columns. Apparently, they can accommodate
limitless number of columns (limited according to implementation) which possibly
made at operating time or at schema definition.

4. Graph-Oriented – This scheme be up to support complex data
errors which are also carry out in an approximately short period of time compared to rest databases
utilizing the schemes
remarked above.

Furthermore, non-relational
databases give high
adjustability for any changes of an attribute by the database because they do not have a
predefined database schema.

 

II.                                                                              COMPARATIV E ANALYSIS ON RDBMS VS.NOSQL.

 

 

Following
points have been summarized from 8 as a part of
providing a comparative analysis on Relational databases and NoSQL data bases:

Transaction
reliability: RDBMS support ACID properties to provide transaction
reliability whereas NoSQL databases are not reliable
like
RDBMSs because of its weaker BASE properties compared to ACID.

Data Model: Relational Databases are
 based 
on  relational model  where
 tables 
that  contain set  of
 rows  represent 
the relation. On the other hand NoSQL databases take many modelling techniques
like
key value stores, document data store, column data store and graph data model

Scalability: The internet based web applications require
horizontal scalability as it spread over several servers in a distributed environment.
NoSQL data store support horizontal scalability whereas it is a great challenge for the relational model.

Cloud:
The relational databases
cannot handle schema less
unstructured data as it can work only with well-defined schema. But it is one of the requirements for handling cloud databases.
However NoSQL
databases
are
fit for the cloud scale solution as it fulfills all of the characteristics
which are desirable
for
cloud databases.

Big data handling: Because of their issues with scalability and data distribution
in a
clustered environment,
it is not an easy task for relational database to handle big data. On the other hand
NoSQL databases designed to handle the big data distributed in the clustered environment.

Complexity: Day by day complexity
in relational databases
rises because of the continuous
rapidly
changed requirements. If the data for the changed requirements does not fit in the
existing
RDBMS schema, then it would make a complex
situation in terms of changing schema and related programming code. On the other hand there is no significant effect on NoSQL databases as they can store unstructured, semi-structured or structured data.

Crash Recovery: Recovery manager ensures crash recovery
for RDBMS data. On the other hand crash recovery depends on data replication for NoSQL databases. MongoDB uses Journal file as recovery mechanism.

Security: Very secure mechanisms are adopted by RDBMSs to secure their data. NoSQL databases are designed for storing and
handling big data, and subsequently providing higher performance at the cost of security. Security of information is a
big
concern of the newly evolving cloud
environment which is being considered as the next generation architecture for enterprises.

In this paper we focus on NoSQL database, namely
MngoDB.This paper  proposed  a  framework  that
 integrates relational database and NoSQL database and handle database
migration. The main fetchers of the framework is as follows.

1.  SQL Interface to relational and NoSQL Database. We offer SQL interface to access both relational
and
NoSQL database. With the use of this interface application does not
need to modify queries or handle NoSQL queries. So actual system design can access both
database.

 

2.  Database Converter. Database migration from relational to
document-oriented
database can be done. Features of
continuous synchronization also there.

 

 

 

III.                                                                             DATA MIGRATION FRAMEWORK

The data migration framework is incredibly modularized, layered between application and databases. It is responsible for performing queries from applications
and
data migration between databases at the same time. The framework provides a SQL interface parsing query statements
to access both a relational database and a
NoSQL database.

We offer a mechanism to control the database migration
process and let applications perform queries whether or not
target data (table) are being migrated. After data migration done, we
 provide  a  patch  mechanism to
 synchronize  inconsistent
tables. We present the data migration framework with its design
and
implementation in following sections.

 

 

 

Application

 

Query result

 

Query

 

 

RDB

2.1  Actual system with Relational database only

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

2.2  System architecture with data migration framework and its
component.

Mediator: Mediator is an interface which accepts queries from
applications, parses queries, extracts and sends necessary
information to controller. Parser can tell the difference between read and write queries and pass the information to controller to put write queries, which might be affected by transformation
progresses, in a
queue if necessary.

Mapping: Hear  one 
SQL queries translator is
 their  which

translate the SQL query into no SQL query. If user wants to fetch the data it directly fetch from NoSQL server.

Meta data store: It store write queries data of appliction Migration tool: extract the data from SQL server transform that data into NoSQL format and then load data into the NoSQL
server

 

Data accumulated
by
information systems is of the
important
assets for many of the businesses. Pushed by customer demands
and force by changes in technologies, companies from time to time migrate from
one information system to  a  different. Hence, knowledge from the  bequest system should be migrated to the new system.

Data migration has two main steps: first, restructuring
of source data according to requirements of the new system, and
second, transferring
data
from the source to the new database.
The academic literature offers many approaches /methods for dealing with these steps: schema conversion,
meta-modeling
approach, ETL (Extract,
Transform, and Load), program
conversion, model-driven
migration, automated data migration.
We use ETL ((Extract, Transform, and Load)
method
for migration relational database to document oriented database. hear flow chart of data migration.

 

 

 

 

 

 

 

 

 

 

2.3 algorithm for migration

 

The execution process of migration of Relational to document
oriented database are as follows.

 

1. Get list of table from Relational database.

 

2. Get all the fields of table.

 

3. Fetch all values from relational databases with PSQL.

 

4.  Format  the
 data  fetch  from
 relational
 data  and
 make compatible with NoSQL databases.

 

5. Check all the type of values is it supported to targeted NoSQL

database?

 

6. If yes then insert data directly
into targeted database. Or do type cast make them suitable to targeted databases.

 

x

Hi!
I'm Erica!

Would you like to get a custom essay? How about receiving a customized one?

Check it out