Articles related to financial technologies (fintech)

Empirica holds workshop on Warsaw Stock Exchange

/in , , , , /by

Algorithmic trading workshop took place on 27th of July 2013 as a part of the second conference held by economic magazine ‘Puls Biznesu’ and Warsaw Stock  Exchange.

Michał Różański, representing Empirica, held workshop on the practical aspects of selecting tools for algorithmic trading by financial institutions. He stressed and covered in detail, especially one aspect of algorithmic trading which is from our practical experience constantly undervalued – namely proper testing of algorithms.

Very interesting was also a lecture of Emil Lewandowski who showed an algorithm which was able to detect a flash crash an hour before it actually happened. Algorithm was implemented, backtested, executed and presented to all the participants our Algorithmic Trading Platform. It was indeed very interesting example of application of algorithmic trading!

Among other guest were representatives from IBM, Sungard, List and M10.

Link to event:

http://konferencje.pb.pl/konferencja/705,handel-algorytmiczny-cz-ii

Artificial intelligence in FinTech

/in , , , , , , , /by

FinTech : It is just starting

FinTech sector is producing businesses with scalable products and has seen rapid growth over the past few years. Senior executives at banks are responding to the challenge these companies have started by setting their own incubators up to capture this high-speed initiation.

Technology was once centralised, with companies being run on big databases and transaction engines. Nowadays, it is massively distributed. New businesses have sprung up to take advantage of the chances this shift brings, while leading banks still operate using the old technology. The term “peer to peer” captures some of the phenomenon, in that it is now potential for financial transactions to take place on a platform without needing a bank or indeed any entity as an intermediary.

The financial services marketplace is all about information exchange that is reliable, secure and efficient. In many cases the new alternatives can be cheaper and quicker than traditional models. A broad variety of potential models exist, which explains the increasing number of new fintech startups that have entered the market.

Needless to say, fintech is not new and technology has consistently brought gains to consumers. Back in the day, however, development costs were high, while the technologies of today are more broadly available, affordable and, most importantly, worldwide scalable.

The huge banks are setting up their own initiation arms to investigate opportunities presented not only mobile but also by by P2P and micro-payments cryptocurrencies like Bitcoin,, and distributed ledgers for example blockchain.

But as progressive as traditional financial institutions strive to be, they will remain hampered by their legacy systems and processes. I see the banking landscape continuing to change quickly as fintech businesses with talented management, viable products and clever advertising using new and traditional media take market share. Moving fast, nimbly and economically to capitalise on opportunities is the key.

Artificial intelligence in FinTech

Since its inception in the 1950s, artificial intelligence (AI) has found at least two major boom cycles and long winters of disillusionment. While artificial intelligence endured through the recent disullusionment cycle in the 1990s to today, its easing and corollary technologies have flourished, and we’re now entering into a fresh boom in applictions of the technology.

Financial services have been revolutioned by the computational arms race of the last twenty-plus years, as technologies like big data analytics, expert systems, neural networks, evolutionary algorithms, machine learning and more have enabled computers to crunch much more varied, diverse, and deep data sets than ever before.

While most of the businesses built around machines making decisions are’t true AI, they may be using data-intensive technologies that will help technologies and firms continue to get closer to executing AI in commercial applications.

Despite the hype of intelligent machines, the first uses of AI are’t replacing humans and human intelligence but augmenting them. Text-based conversational chat was adopted by many startups as a way to deliver a personal assistant-like expertise in many industries, and in fintech we’ve seen the case of businesses like Kasisto utilising AI to scale the impact of people using technology. Instead of being bounded in customer support uses by humans reacting to users through chat windows, AI and related technologies are being implemented to deliver a human-like chat encounter without the need for nearly as many human helpers.

By using smart agents that can examine and crunch data about individual behaviour and compare to broader datasets, small and big businesses could have the ability to deliver personalized financial services as a scope and scale never possible before. Consumer banking, advisory services, retail financial planning, investment advice and wealth management, all of these services can be delivered using a conversational user interface with artifical intelligence software behind. The combination of technologies can empower firms to supply services to customers where they were unable to supply human service profitably (i.e. lower net worth sections of personal financial, investment and retirement advisory), but can now function using codified knowledge and AI-powered software.

In addition to new segments, they are able to be more personal, supplying guidance at the transactional level (i.e. every individual transaction). This is the story behind smart wallets like Wallet.ai. Picture having an assistant with you to allow you to assess, price, and consider every single thing you spend money on, at a granular level that you could not be assisted by any human helper with. Is a roboadvisor that offers rule based advice using only a couple of predefined parameters AI? Likely not, but newer technologies as time goes by which are based around learning and viewing about your behaviors at the individual level, could give guidance and outcomes which might be personalized in a way never possible formerly.

AI can also power technologies that overlay humans to supply workers activities with an tracking and oversight mechanism, helping with compliance, security, and the observation of employee actions. Monitoring discrete, repetitive data entry tasks, computers could watch and learn as time passes to verify test and data entry for particular events, evaluate danger, and find fraud. Any segment of fintech that is regulated creates the chance for companies to install AI-powered employee and systems supervision.

AI technologies that allow computers to process information could augment underwriting and lending products and make decisions more easy and better than individuals alone. While it’s still to be determined how new data sets created by technologies like wearables and internet of things can be used for insurance and credit decisions, AI-based technologies make it more potential for businesses to use these new datasets in highly personal ways .

But AI is creating bigger opportunities to go beyond testing and fitting data to create trading systems and more “intelligent ” dealers, using robotraders to optimize and test predictions and trading rules. AI can help manage and augment rules and trading decisions, helping process the data and creating the algorithms managing trading rules.

Some investment firms have implemented trading algorithms based on sentiment and insights from social media and other public data sources for years, but technology companies like Dataminr are installing platforms for a larger set of businesses to use. Getting and utilizing large, heterogenous datasets is now potential for far more companies to use, so how will companies leverage and build on top of these datasets?

The future of AI in FinTech

While much of the investment in artificial intelligence has been into multi-purpose platforms which are figuring out their specific, high-value usecases, the chance in fintech is somewhat different. Fintech has a base of technological prowess in the technologies supporting AI and several immediate high value uses.

Initially, AI was used more in backend technology settings to power large scale decisioning in financial analysis , trading and lending, but now it is becoming a technology that expands how everybody interacts with financial services companies. A number of problems consumers are facing when using financial services are around the problems in getting to quality, personal service. And possibly it’s an artificially intelligent agent that helps deliver cheaper, private services that are better and faster.

Empirica with lecture at ‘Algorithmic Trading Conference’

/in , , , /by

Conference on the subject of ‘Algorithmic Trading’ was held at Warsaw Stock Exchange headquarters on the 28th of February 2013. The event was open by the WSE president, Adam Maciejewski. Among the invited guests were:

  • Peter Van Kleef, Lakeview Capital president
  • Michal Rozanski, CEO of Empirica
  • Andrzej Endler, CEO of M10
  • Michal Kobza, Warsaw Stock Exchange.

Michal Rozanski from Empirica made lecture on topic ‘Tools supporting financial institutions in algorithmic trading’. He showed not only common functionalities and architectures of available solutions, but also talked about practical aspects of hard decision every financial institution faces – to build software tools by own IT department or to buy from external vendors.

Very interesting was lecture held by Peter Van Kleef. Among other topics he shared his experiences from high frequency trading and how it has changed during last years.

We have informations that organizators intend to prepare soon another event relating to topic of algorithmic trading.

Link: GPW conference

Apache Spark Software Development FinTech

Apache Spark – fast big data processing

/in , , , /by
Businesses are using Hadoop widely to analyze their data sets. The reason is that Hadoop framework is founded on a straightforward programming model (MapReduce) and it enables a computing solution that is scalable, flexible, fault-tolerant and economical. Here, the main issue would be to keep speed in waiting time to run the program and processing large datasets in terms of waiting time between queries.
Apache Software Foundation for speeding up the Hadoop computational computing software process introduced Spark.
As against a standard idea, Spark is not a modified version of Hadoop and is not, really, dependent on Hadoop because it’s its own cluster direction. Hadoop is only one of the methods to implement Spark.
Spark uses Hadoop in two ways – one is second and storage . It uses Hadoop for storage function only since Spark has its own cluster direction computation.

Apache Spark

Apache Spark is a lightning-quick cluster computing technology, designed for fast computation. It’s based on Hadoop MapReduce and it expands it to be economically used by the MapReduce version for more types of computations, including interactive queries and stream processing. The principal feature of Spark is its in-memory cluster computing that increases the processing speed of an application.

 

Spark was created to cover a wide variety of workloads such as streaming, iterative algorithms, interactive queries and batch applications. It reduces the management burden of maintaining different tools, besides supporting all these workload in a specific system.

Evolution of Apache Spark

Spark is one of Hadoop’s sub project developed in 2009 in UC Berkeley’s AMPLab by Matei Zaharia. It was Open Sourced in 2010 under a BSD license. It was given to Apache software foundation in 2013, and Apache Spark has become a top level Apache project from Feb-2014.

 

Characteristics of Apache Spark

Apache Spark has following attributes.

 

Speed − Spark helps to run an application in Hadoop cluster, 10 times faster when running on disc, and up to 100 times faster in memory. This is possible by reducing amount of read/write operations to disk. It stores the intermediate processing data in memory.

 

Supports multiple languages − Spark provides built in APIs in Java, Scala, or Python. Consequently can write applications in distinct languages. Spark comes up with 80 high level operators for interactive querying.

 

Advanced Analytics − Spark supports ‘Map’ and ‘ reduce’. Additionally, it supports SQL queries, Streaming information, Machine learning (ML), and Graph algorithms.

Spark Assembled on Hadoop

The following diagram shows three ways of how Spark can be assembled with Hadoop parts.

 

Ignite Constructed on Hadoop

There are three manners of Spark installation as described below.

 

Standalone − Spark Standalone deployment means Spark inhabits the place on top of HDFS(Hadoop Distributed File System) and space is allocated for HDFS, explicitly. Here, MapReduce and Spark will run side by side to cover all spark occupations on bunch.

 

Hadoop Yarn − Hadoop Yarn deployment means, only, spark runs with no pre-installation or root access needed on Yarn. It helps to integrate Spark into Hadoop stack or Hadoop ecosystem. It enables other components to run in addition to stack.

 

Spark in MapReduce (SIMR) − Spark in MapReduce is used to launch spark occupation in addition to standalone deployment. With SIMR, Spark can be started by user and uses its shell with no administrative access.

Parts of Spark

Apache Spark Software Development FinTech

Apache Spark Core

Spark Core is the inherent general execution engine for Spark platform that all other functionality is built upon. It provides In-Memory computing and referencing datasets in external storage systems.

 

Spark SQL

Spark SQL is a part in addition to Spark Core that introduces a new data abstraction called SchemaRDD, which provides support for structured and semi-structured data.

 

Start Streaming

Spark Streaming leverages Spark Core’s fast scheduling capability to perform streaming analytics. It ingests info in mini-batches and performs RDD (Bouncy Distributed Datasets) transformations on those mini-batches of data.

 

MLlib (Machine Learning Library)

MLlib is a distributed machine learning framework above Spark due to the distributed memory-based Spark design. It is, according to benchmarks, done by the MLlib developers against the Alternating Least Squares (ALS) enactments. Spark MLlib is nine times as rapid as the Hadoop disk-based version of Apache Mahout (before Mahout gained a Spark interface).

 

GraphX

GraphX is a distributed graph-processing framework in addition to Spark. It supplies an API for expressing graph computation that can model the user- . Additionally, it provides an optimized runtime for this abstraction.

 

Spark vs Hadoop

 

Listen in on any conversation about big data, and you’ll probably hear mention of Hadoop or Apache Spark. Here is a brief look at what they do and how they compare.

 

1. They do different things. Hadoop and Apache Spark are both big-data frameworks, but they don’t actually serve the same functions. Hadoop is basically a distributed information infrastructure: It doles out huge data collections across multiple nodes within a cluster of commodity servers, which means you do not need to buy and keep expensive custom hardware. In addition, it indexes and keeps track of that info, empowering big-data analytics and processing far more effectively than was possible previously. Spark, on the other hand, is a data-processing tool that operates on those distributed data collections; it doesn’t do distributed storage.

 

2.  You can use one without the other. Hadoop includes not only a storage part, referred to as the Hadoop Distributed File System, so you don’t need Spark to get your processing, but also a processing part called MapReduce. Conversely, you can even use Spark without Hadoop. Spark does not come with its own file management system, though, so it must be integrated with one — if not HDFS, afterward another cloud-based info platform. Spark was designed for Hadoop, however, so many agree they’re better collectively.

 

3. Spark is quicker. Spark is usually a lot quicker than MapReduce because of the way it processes data. Spark functions on the entire data set in one fell swoop while MapReduce operates in measures. “The MapReduce workflow looks like this: read information from the cluster, perform an operation, write results to the cluster, read updated information from the cluster, perform next operation, write next results to the bunch, etc.,” explained Kirk Borne, principal info scientist at Booz Allen Hamilton. Spark, on the other hand, finishes the full data analytics operations in-memory and in near real time: “Read information from the bunch, perform all the necessary analytic operations, write results to the cluster, done,” Borne said. Spark can be as much as 10 times quicker than MapReduce for batch processing and up to 100 times quicker for in-memory analytics, he said.

 

 

4. You may not need Spark’s speed. MapReduce’s processing fashion can be just fine if your data operations and reporting conditions are mostly static and you’ll be able to wait for batch-mode processing. But if you must do analytics on streaming data, like from detectors on a factory floor, or have applications that need multiple operations, you probably want to go with Spark. Most machine learning algorithms, by way of example, need multiple procedures. Common uses for Spark contain real-time marketing campaigns, product recommendations that are online, cybersecurity analytics and machine log monitoring.