What is Big Data?

First of let’s start off by defining what is the term “Big Data”? Big Data has truly come of age in 2013 when Oxford English Dictionary introduced the term “Big Data” for the first time in its dictionary REF1. The term refers to the data that we transmit to the internet Like social media, healthcare sector financial, etc. The question here How big is this data? According to the MicroFocus REF2 each minute every day the following data is being generated :

  • 4.3 BILLION Facebook messages posted daily!
  • 5.75 BILLION Facebook likes every day.
  • 22 billion texts are sent every day.
  • 5.2 BILLION daily Google Searches in 2017.
  • Instagram users upload 46,740 million posts every minute!
  • Youtube usage more than tripled from 2014-2016 with users uploading 400 hours of new video each minute of every day! Now, in 2017, users are watching 4,146,600 videos every minute.

So three characteristics that form the big data it is called the three V :

  • Volume: Represent the amount of data
  • Velocity: Speed of the data stream
  • Variety: The different types of data

But the question is how is this data being stored? Basically big data comes in 3 forms :

  • Structured: Which comes in form of a Database or Data Warehouse. This form of data is being stored using some special software for storing Structured Data Like Oracle • MySQL • PostgreSQL• MsSQL
  • Semi-Structured: These forms of data are represented in XML or even emails. This data is stored in XML or JSON format.
  • Unstructured: These data are the data on the internet that are not structured or archived Like GPS data, data collected from social media, and more. MongoDB, Cassandra, Neo4j, and Redis are NoSQL software to store Unstructured Data.

Big Data represented into two types

  • Data in movement: Twitter, Facebook comment, and more.
  • Data at rest: Email logs, Forms, and anything that is being stored.

How can we use these data to improve our life? How does it affect our future In this document we will discuss 3 applications the Big Data that could affect and improve our lives. We start off with :

Wearables 

A fitness tracker or any bracelet equipped with a heartbeat or movement sensor. Using the data transmitted by the bracelet we can tell if the person is healthy, active, or even lazy. We can understand his/her active hour or if he/she even had enough sleep. Furthermore, with enough heartbeat data, we can even predicate if he/she has any medical problem in the near future.

Online Shopping 

Big Data is also being used in online shopping to help the system suggest you a product or even make a discount for loyal customers. This will have a big impact on the customer experience since the customer will get what he/she prefers displayed dynamically for him/her. This will result in the customer spending more money on stuff they want and like. Keeping in mind using the big data we collect we can reward the loyal customers by giving them, For Example, limited-time discount and offering them bundles specially made for them using previously collected data.

City Infrastructure

Smart cities heavily depend on big data and recent analysis to ensure smooth and precise prediction. For example, predicting the traffic so you adjust the plane schedule or adjust public transportation plan for better efficiency. Also, we can use it for better estimation time for reaching a specific destination. Furthermore, it helps the Police and Tow companies to predict the rate of accidents So the city can get ready for it to reduce the dispatch time which will result to save lives. The applications will go on.

But with the benefits comes security concerns. 

One of the big disadvantages of big data is security. Security breaches are a big concern these days. Hence paying enough attention and investing enough in security is so essential So we can benefit from big data without affecting our life negatively since these data sometimes could be so sensitive.   

  In conclusion, Big Data is definitely affecting the future of our life and evolving the technology we use at a fast pace in a beneficial way. In my opinion, we will see more companies adopting the idea of investing in Big Data applications when there is enough support for it in a way that companies can start profiting from it. Basically, in the current future, it is really expensive and costly to start changing the current companies to adopt the idea of big data and deploy its applications. Hence it is easier for new companies to build upon it. Moreover, we need to keep in mind the security risk the companies have to take in storing and maintaining the data which is a big trust issue, and the very sensitive role the companies have to play.

REF1 https://www.kdnuggets.com/2017/02/origins-big-data.html

REF2 https://blog.microfocus.com/how-much-data-is-created-on-the-internet-each-day/

Cloud and Cloud Security In Banking Sector

What is the cloud?

Cloud can be described as a big interconnected network of computational power. Cloud can be utilized in different ways for example Cloud Storage and Cloud Computing. According to Wikipedia RF1, Cloud Computing can provide its services in different ways Like Infrastructure as a service (IaaS), Platform as a service (PaaS), and Software as a service (SaaS). Why it is beneficial for companies to use clouds? Companies can depend heavily on cloud services for the massive benefits that the companies can utilize from it like:

Fast and easy deployment :

When using cloud-based services deploying a new product to keep up the market is never been easier since as an IT department, you don’t have to worry about the hardware requirement. Hence you can cut down the cost of buying new infrastructure or even doing a simulation to measure if the existing hardware is enough for you. The cloud should be able to distribute the load among its nodes.

Cut Down The Cost:

Since cloud services are based on the disturbing load you don’t have the problem of having too much or too little resources as you can adjust your needed resources. Furthermore, some services let you pay as much as you use which could save more money. In addition, we should consider the cost we are saving of maintaining the servers in case of not using the cloud services. Companies can take advantage of the cloud without having to manage and invest upfront capital in a complex on-premise deployment.

Bandwidth limitation :

As a big corporation network bandwidth is really important since the last thing you want is to have your network experiencing bandwidth overhead. This will result in a very slow response time. Instead, we use the cloud to distribute the load over the nodes and have a much more bandwidth limit. In this way, you move the load away from the company servers which will save money and time. Thus moving big data will be much faster, easier, and more reliable.

Redundancy  :

Data is the infrastructure that the companies operate. So having a backup data center is essential for any company. But data centers are really expensive to build and maintain. Having a backup data center up and running all the time synchronized with your primary data center is not an easy and cheap task. Therefore having a cloud storage service hosting your data seems like a good idea especially that so reputable cloud storage provider will always ensure that your data is stored on multiple machines and in compliance with any regulations that apply to your industry (for example, some businesses may be required to have up to three copies of all data). But there’s more to it than duplicating information RF2.

Ease of use :

In some start-up companies, they don’t have to budget to build and configure their own servers. Therefore using a pre-configured service would save money and time to quickly use the service without having to worry about configuration. Furthermore some cloud services providers Like S3 “Cloud Storage” provides an easy cross-platform GUI which comes at a really reasonable price for the business. Even more, some big- corporations may need to add or remove infrastructure to meet specific demands, such as RF3

  • Seasonal customer traffic loads
  • Sudden “crunch time” IT demands
  • Growth of business staff using your company’s apps

If you’re relying on in-house IT infrastructure, you won’t have the flexibility needed to make quick changes to your infrastructure.

The Theory

How the cloud could be beneficial in the banking sector ?RF4

Considering most banks have multiple branches doing the same transaction, again and again, IT can use cloud as a central server so every bank can send their transaction to be performed in the cloud. Furthermore, some cloud services come with built-in compliance and security-sensitive financial workloads are no problem for maintaining efficient deployment and operations. Hence the user can communicate with the server using the API which will provide support for 3rd party apps to give the user more flexibility. 

Windows Azure is providing its services to the banking sector to benefit from T24 as a software as a service platform SaaS. Temenos with its services called T24 on the Windows Azure platform will allow banks to move operations to a consumption-based pricing model and enables them to scale resources effectively and increase volume according to customer demand.

What is the benefit of using the cloud as a central server for all transactions?

Flexibility and scalability: Cloud will allow the banks to adjust their resources according to the market need either to upscale or downscale. The ability to respond quickly will be an important competitive edge.

Increase efficiency: Using the cloud will give the ability to the banks to adopt and integrate new technologies very fast to fulfill their customer’s needs. Serve clients faster: When using cloud services it’s faster to develop and deploy new apps.

Cut down the cost: being able to use a consumption-based pricing model will save money for the banks during low load days instead of having the upfront investment for a resource you might not need during the whole year except seasons time.
Redundancy: As we mentioned before that redundancy is crucial in the banking sector cloud will guarantee you almost 100% UPTIME depending on the provider. So reputable providers might even provide 2 – 3 duplicates. Hence your bank will never miss any important transaction.

How To Install Linux, Apache, MySQL, PHP (LAMP)


Setting up Apache Web Server

Install Apache

First thing first install Apache2 using the following command :

sudo apt-get install apache2
?
Testing Apache

You can test if the installation successfully by visiting http://locahost/ from the machine itself or the local ip of the machine.

Changing The Default Web Page

You can nano -> /var/www/html/index.html to change the default page using the following code :

nano /var/www/html/index.html

Once you are done Ctrl+X and Enter to save your work.

Install PHP

Install the PHP and Apache packages with the following command:

sudo apt-get install php5 libapache2-mod-php5 -y
Test PHP

You can test if the PHP installation went correctly by creating a PHP file and visit it using the following :

cd /var/www/html/

sudo rm index.html

nano ./index.php
?

Once it is open paste this code :

<?php echo "Hello World"; ?>

Save your work using Ctrl+x hit Enter. Write the following code :

sudo service apache2 restart

Then from the machine itself navigate to http://localhost/index.php OR use the machine local IP instead of localhost. If you can see the message Hello World then PHP is running correctly.

Install MySQL

Install the MySQL Server and PHP-MySQL using the following command :

sudo apt-get install mysql-server php5-mysql -y

When installing MySQL you will be asked for a root password. You’ll need to remember this to allow your website to access the database.

Now restart Apache:

sudo service apache2 restart

And that that is it for today, Now you a fully functioning LAMP server that you can access internally.

7 Important Linux Commands


Here are 7 important Linux commands to use during your SSH session.

Move Command:

Move command is used to move files around or even rename files, here’s the generic form:

“ mv [options] SOURCE DESTINATION “

Let’s say we want to move FILE1 to the previous FOLDER

“ mv FILE1.TXT .. “

or we want to rename FILE1 to X

“ mv FILE1.TXT ./X.TXT

ok let’s make it even harder and let’s say we want to move everything to the previous folder:

“ mv * ../ “

NOTE: * mean everything in this particular example it meant everything in this folder.

Change Directory:

cd is a simple command to change directory, Here’s the generic form:

“ cd DESTINATION “

For example, we want to move to /home/var/www

“ cd /home/var/www “

ok now let say we want to move to a folder inside the current folder:

“ cd ./FOLDER-NAME “

or even move to the previous folder simply write:

“ cd .. “

Simple ! huh ?

Remove File Or Directory:

rm command is used to remove files, and in order to remove folders you have to include -rf for example :

“ rm * “

The previous command will actually remove all files if the current folder contains folders as well you need to do the following

“ rm -rf * “

This command will result in removing everything in the current folder. Keep in mind you can also remove the and specify a folder name instead.

NOTE: In some cases, you might need to use SUDO to be able to execute this command.

List Directory Content:

ls is used to list all files and folders inside the current folder.

“ ls “

we are not going to go into the details of this command since it is mostly used as demonstrated above.

Change Mode Command:

CHMOD or Change mode command is used to change permissions of files or folder “Directory”. It is crucial if you want to give a system a power to do things or even restrict it.

“ chmod XXX NAME-HERE “

you replace XXX with the permission number. There is also another variation for changing the mode recursively as following

“ chmod -R XXX FOLDER-NAME “

Web Get :

Web Get or wget command is used to download stuff from the internet directly to your machine instead of using FTP or SFTP. In some cases, you need to install it using your package manager depending on your OS. For Ubuntu or other Debian based distribution you can install it using the following command:

“ sudo apt-get install wget “

After that, you are ready to go. Here are some examples:

“ wget URL “

It is preferred to be a compressed file Like .ZIP So you can unzip it easily. It is used for example to download WordPress files for any other purpose.

UNZIP Command:

UNZIP command it’s simply a command to “unzip” or uncompress files from a ZIPPED file. Let’s say you got a file from wget command and you want to unzip in the current folder you do the

following:

“ unzip FILE-NAME.ZIP “

If you don’t know the file name you just use the command above “ ls “. You can also specify the destination using:

“ unzip FILE-NAME.ZIP -d DESTINATION “

NOTE: In some cases, you need to install it using your package manager depending on your OS. For Ubuntu or other Debian based distribution, you can install it using the following

command:

“ sudo apt-get install unzip “

Finally, I would like to note that some commands mentioned above have some other options or variations. This is just a basic guide to get you started with your VPS or SERVER.

4 essential tools for Linux

Rsync

Rsync is an open source bandwidth-friendly utility tool for performing swift incremental file transfers and it is available for free.

$ rsync [OPTION...] SRC... [DEST]
?

Install Rsync on Ubuntu and Debian, using the following commands.

$ sudo apt-get install rsync
$ sudo apt update

Vim

Vim is an open source clone of vi text editor developed to be customizable and able to work with any type of text.

Install Vim on Ubuntu and Debian, using the following commands.

$ sudo add-apt-repository ppa:jonathonf/vim
$ sudo apt update
$ sudo apt install vim

GNOME

GNOME is a Desktop Environment comprised of several free and open-source applications and can run on any Linux distro and on most BSD derivatives.

Install Gnome desktop on Ubuntu and Debian, using the following commands.

$ sudo apt install tasksel
$ sudo apt update
$ sudo tasksel install ubuntu-desktop

Screen Linux

screen command in Linux provides the ability to launch and use multiple shell sessions from a single ssh session. When a process is started with ‘screen‘, the process can be detached from session & then can reattach the session at a later time.

Install Screen desktop on Ubuntu and Debian, using the following commands.

$ sudo apt install screen
$ sudo apt update