Monday, 4 December 2017

Bank Account Statement transactions management using Scala type classes, Shapeless

Untitled Document.md

Bank-Account-Typeclasses

Bank Account management using scala type classes and Shapeless.

A simple Bank Account contains statements of type credit (Income)/ debit(expenses). The statement is generated from last balance by adding all credit transactions and subtracting all the debit transactions.
Many ways we can summarize the statements,

Simple Functional way to generate summary is,

sealed abstract class Transaction(val price: Int, description: String)
final case class Income(override val price: Int, description: String) extends Transaction(price, description)
final case class Expenses (override val price: Int, description: String) extends Transaction(price, description)
final case class Transactions(items: List[Transaction])

  def generateSummary(oldBalance: Int, txns: Transactions): Int = {
    var newBalance = oldBalance
    txns.items.foreach(item => {
      item match {
        case Income(price,_) => (newBalance += price)
        case Expenses(price, _) => (newBalance -= price)
      }
    })
    newBalance
  }

Issues with the above approach

Is that for example if we want to add new transaction WithHeld (for example a fraudulent transaction should not be included in the summary, then we need to modify the generateSummary function to include the new case to ignore the transaction as below,

final case class WithHeld (override val price: Int, description: String) extends Transaction(price, description)
  def generateSummary(oldBalance: Int, txns: Transactions): Int = {
    var newBalance = oldBalance
    txns.items.foreach(item => {
      item match {
        case Income(price,_) => (newBalance += price)
        case Expenses(price, _) => (newBalance -= price)
        case WithHeld(price,_) => (newBalance)
      }
    })
    newBalance
  }

Type classes

Instead of calculating the summary in one method for all transaction type, if we add behaviour to each transaction type on how to change balance, then for new type it would just be adding the behaviour for the new type and using Scala’s implicit and Shapeless HList and Coproduct, we can easily process for all the transactions.

For Type classes, three separate components glued together,

1. Api represents functionality to be implemented for the types.
2. Instances of the type classes for the types which we care about
3. Interface exposed to the user.

API:

In BankAccount statement generation, we need functionality to accumulate each type of transaction to the current balance.

input (Int, T) - (old balance, and the Type for which the balance to be accumulated).

trait Accumulator[T] {
  def accumulate(input: (Int, T)): (Int)
}

Instances we care about:

  implicit val  incomeAccumulator = new Accumulator[Income] {
    override def accumulate(input: (Int, Income)) = {
      (input._1 + input._2.price)
    }
  }

  implicit val  expenseAccumulator = new Accumulator[Expenses] {
    override def accumulate(input: (Int, Expenses)) = {
      (input._1 - input._2.price)
    }
  }

Interface Exposed to the user:

  def generateStatementSummary[A](value: (Int, A))(implicit enc: Accumulator[A]): Int =
    enc.accumulate(value)

Calculating summary

To calculate Summary, simply calling

  val statements = Transactions(List(Income(3000, "Salary")))
  val oldBalance = 4000
  val newBalance = generateStatementSummary(oldBalance, statements)
  //newBalance would be 7000

New Transaction type

If we have to add new transaction type WithHeld, then simply

//Defining new Type
final case class WithHeld (override val price: Int, description: String) extends Transaction(price, description)

//And adding behaviour to new type

  implicit val  withHeldAccumulator = new Accumulator[WithHeld] {
    override def accumulate(input: (Int, WithHeld)) = {
      (input._1)
    }
  }
  

Would be enough to calculate the statement summary generation and we don’t need to change the implementation of generateStatementSummary api.

And now

  val statements = Transactions(List(Income(3000, "Salary"), WithHeld(400, "Fradulent transaction, withheld"))
  val oldBalance = 4000
  val newBalance = generateStatementSummary(oldBalance, statements)
  //newBalance would be 7000

would use the WithHeld behaviour using the newly defined implicits and return the same balance.

Credits to the below links:

1. https://github.com/milessabin/shapeless 
2. http://gigiigig.github.io/tlp-step-by-step/dependent-types.html
3. http://danielwestheide.com/blog/2013/02/06/the-neophytes-guide-to-scala-part-12-type-classes.html
4. https://github.com/underscoreio/shapeless-guide

Saturday, 3 May 2014

Transaction Isolation:


The ANSI/ISO SQL standard defines four levels of transaction isolation, with different possible outcomes for the same transaction scenario. That is, the same work performed in the same fashion with the same inputs may result in different answers, depending on your isolation level. These levels are defined in terms of three phenomena that are either permitted or not at a given isolation level:

    Dirty read: The meaning of this term is as bad as it sounds. You're permitted to read uncommitted, or dirty , data. You can achieve this effect by just opening an OS file that someone else is writing and reading whatever data happens to be there. Data integrity is compromised, foreign keys are violated, and unique constraints are ignored.

    Nonrepeatable read: This simply means that if you read a row at time T1 and try to reread that row at time T2, the row may have changed. It may have disappeared, it may have been updated, and so on.

    Phantom read: This means that if you execute a query at time T1 and re-execute it at time T2, additional rows may have been added to the database, which may affect your results. This differs from a nonrepeatable read in that with a phantom read, data you already read hasn't been changed, but instead, more data satisfies your query criteria than before.

Note that the ANSI/ISO SQL standard defines transaction-level characteristics, not just individual statement-by-statement-level characteristics. I'll examine transaction-level isolation, not just statement-level isolation.

The SQL isolation levels are defined based on whether they allow each of the preceding phenomena. It's interesting to note that the SQL standard doesn't impose a specific locking scheme or mandate particular behaviors, but rather describes these isolation levels in terms of these phenomena—allowing for many different locking/concurrency mechanisms to exist (see Table 1).



Isolation Level                   Dirty Read     Nonrepeatable Read     Phantom Read

READ UNCOMMITTED      Permitted      Permitted                        Permitted
READ COMMITTED                   -             Permitted                        Permitted
REPEATABLE READ                  -                   -                                 Permitted
SERIALIZABLE                          -                   -                                     -


Serializable (From Wiki)

This is the highest isolation level.

With a lock-based concurrency control DBMS implementation, serializability requires read and write locks (acquired on selected data) to be released at the end of the transaction. Also range-locks must be acquired when a SELECT query uses a ranged WHERE clause, especially to avoid the phantom reads phenomenon (see below).

When using non-lock based concurrency control, no locks are acquired; however, if the system detects a write collision among several concurrent transactions, only one of them is allowed to commit. See snapshot isolation for more details on this topic.



Repeatable reads

In this isolation level, a lock-based concurrency control DBMS implementation keeps read and write locks (acquired on selected data) until the end of the transaction. However, range-locks are not managed, so the phantom reads phenomenon can occur (see below).

Read committed

In this isolation level, a lock-based concurrency control DBMS implementation keeps write locks (acquired on selected data) until the end of the transaction, but read locks are released as soon as the SELECT operation is performed (so the non-repeatable reads phenomenon can occur in this isolation level, as discussed below). As in the previous level, range-locks are not managed.

Putting it in simpler words, read committed is an isolation level that guarantees that any data read is committed at the moment it is read. It simply restricts the reader from seeing any intermediate, uncommitted, 'dirty' read. It makes no promise whatsoever that if the transaction re-issues the read, it will find the same data; data is free to change after it is read.
Read uncommitted

This is the lowest isolation level. In this level, dirty reads are allowed, so one transaction may see not-yet-committed changes made by other transactions.

Since each isolation level is stronger than those below, in that no higher isolation level allows an action forbidden by a lower one, the standard permits a DBMS to run a transaction at an isolation level stronger than that requested (e.g., a "Read committed" transaction may actually be performed at a "Repeatable read" isolation level).

Wednesday, 8 August 2012

JAXB-Marshalling-UnMarshalling-using-Generics







JAXB Marshalling / UnMarshalling using Generics:


Java Architecture for XML Binding used to convert XML to JAVA POJO objects. Traditional way is to write SAX / DOM parser and convert that into JAVA Objects, with JAXB the conversion is done in two stages.
    1. Generation of POJO classes from XML Schema Definition
    2. Using JAXB UnMarshaller to convert the XML to POJO objects.


Sample XML to unmarshall the xml to JAVA POJO is,

 public Orders unMarshal(String content) {
  try {
   JAXBContext jc = JAXBContext.newInstance( Orders.class );
   Unmarshaller u = jc.createUnmarshaller();
   return (Orders) u.unmarshal(new StreamSource( new StringReader( content )), Orders.class).getValue();
  } catch (JAXBException e) {
   log.error(String.format("Exception while unmarshalling: %s", e.getMessage()));
  }  
  return null;
 }
In the above code, Orders.class is a generated POJO from below XML schema definition,

<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
            xmlns:jxb="http://java.sun.com/xml/ns/jaxb"
            jxb:version="2.0">

<xsd:element name="Orders">
<xsd:complexType>
  <xsd:sequence>
    <xsd:element name="Order" type="OrderType"
                 maxOccurs="unbounded"/>
  </xsd:sequence>
</xsd:complexType>
</xsd:element>

<xsd:complexType name="OrderType">
  <xsd:sequence>
    <xsd:element name="symbol" type="xsd:string"/>
    <xsd:element name="size" type="xsd:int"/>
    <xsd:element name="executionType" type="ExecutionType"/>
  </xsd:sequence>
</xsd:complexType>

<xsd:simpleType name="ExecutionType">
  <xsd:restriction base="xsd:string">
    <xsd:enumeration value="MARKET_ORDER"/>
    <xsd:enumeration value="LIMIT_ORDER"/>
    <xsd:enumeration value="STOP_ORDER"/>
    <xsd:enumeration value="STOP_LIMIT_ORDER"/>
  </xsd:restriction>
</xsd:simpleType>
</xsd:schema>
If we have to Unmarshall another xml schema CancelOrders,

<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
            xmlns:jxb="http://java.sun.com/xml/ns/jaxb"
            jxb:version="2.0">

<xsd:element name="CancelOrder">
<xsd:complexType>
  <xsd:sequence>
    <xsd:element name="orderNumber" type="xsd:int"/>
    <xsd:element name="reason" type="xsd:string"/>
  </xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema>
 
then the code to unmarshall the object would be,

 public CancelOrders unMarshal(String content) {
  try {
   JAXBContext jc = JAXBContext.newInstance( CancelOrders.class );
   Unmarshaller u = jc.createUnmarshaller();
   return (Orders) u.unmarshal(new StreamSource( new StringReader( content )), CancelOrders.class).getValue();
  } catch (JAXBException e) {
   log.error(String.format("Exception while unmarshalling: %s", e.getMessage()));
  }  
  return null;
 }
Using generics, the above Unmarshalling JAVA code could be simplified and could be reused as below,

 public <T> T unMarshal(String content, Class<T> clasz) {
  try {
   JAXBContext jc = JAXBContext.newInstance( clasz );
   Unmarshaller u = jc.createUnmarshaller();
   return u.unmarshal(new StreamSource( new StringReader( content )), clasz).getValue();
  } catch (JAXBException e) {
   log.error(String.format("Exception while unmarshalling: %s", e.getMessage()));
  }  
  return null;
 }
The above function uses generics and accepts the xml content as string and the POJO class to be converted as generic class.
Using generics, the same code is reused for any xml content to be converted to their specific POJO objects.

Similarly Marshalling of JAVA pojo's to XML cound done using JAVA Generics as below,
 public <T> String marshal(T object) {
  try {
   StringWriter stringWriter = new StringWriter();
   JAXBContext jc = JAXBContext.newInstance( object.getClass());
   Marshaller m = jc.createMarshaller();
   m.marshal(object, stringWriter);
   return stringWriter.toString();
  } catch (JAXBException e) {
   log.error(String.format("Exception while marshalling: %s", e.getMessage()));
  }
  return null;
 }