Thursday, December 31, 2015
yield Keyword Explained
When you use the yield keyword in a statement, you indicate that the method, operator, or get accessor in which it appears is an iterator. Using yield to define an iterator removes the need for an explicit extra class when you implement the IEnumerable and IEnumerator pattern for a custom collection type. The yield keyword was introduced in C# 2.0 (it is not currently available in VB.NET) and is useful for iteration. In iterator blocks it can be used to get the next value.
“Yield keyword helps us to do custom stateful iteration over .NET collections.” There are two scenarios where “yield” keyword is useful:
- Customized iteration through a collection without creating a temporary collection
- Stateful iteration
It is a contextual keyword used in iterator methods in C#. Basically, it has two use cases:
- yield return obj; returns the next item in the sequence.
- yield break; stops returning sequence elements (this happens automatically if control reaches the end of the iterator method body).
When you use the "yield return" keyphrase, .NET is wiring up a whole bunch of plumbing code for you, but for now you can pretend it's magic. When you start to loop in the calling code, this function actually gets called over and over again, but each time it resumes execution where it left off.
Typical Implementation
Typical Implementation
- Caller calls function
- Function executes and returns list
- Caller uses list
- Caller calls function
- Caller requests item
- Next item returned
- Goto step #2
Although the execution of the yield implementation is a little more complicated, what we end up with is an implementation that "pulls" items one at a time instead of having to build an entire list before returning to the client.
Example:
Example:
public class PowersOf2 { static void Main() { // Display powers of 2 up to the exponent of 8: foreach (int i in Power(2, 8)) { Console.Write("{0} ", i); } } public static System.Collections.Generic.IEnumerable<int> Power(int number, int exponent) { int result = 1; for (int i = 0; i < exponent; i++) { result = result * number; yield return result; } } // Output: 2 4 8 16 32 64 128 256 }
Wednesday, December 30, 2015
RabbitMQ : Tackling the broker SPOF
Though RabbitMQ brokers are extremely stable, a crash is always possible. Losing an instance altogether due to a virtual instance glitch is a likely possibility that can't be ignored. Therefore, it is essential to tackle the broker single point of failure (SPOF) before something bad happens, to prevent losing data or annoying users.The RabbitMQ can easily be configured to run in an active/active deployment, where several brokers are engaged in a cluster to act as a single highly-available AMQP middleware. The active/active aspect is essential, because it means that no manual fail-over operation is needed if one broker goes down.
Mirroring queues
With clustering, the configuration gets synchronized across all RabbitMQ nodes. This means that clients can now connect to one node or the other and find the exchanges and queues they're expecting. However, there is one thing that is not carried over the cluster by default: the messages themselves. By default, queue data is local to a particular node. When a queue is mirrored, its instances across the network organize themselves around one master and several slaves. All interaction (message queuing and dequeuing) happens with the master; the slaves receive the updates via synchronization over the cluster. If you interact with a node that hosts a slave queue, the interaction would actually be forwarded across the cluster to the master and then synchronized back to the slave.
Activating queue mirroring is done via a policy that is applied to each queue concerned.
Federating brokers
If you think beyond the notion of a single centralized enterprise resource and instead think in terms of distributed components, the idea of creating a topology of RabbitMQ brokers will emerge. RabbitMQ offers the following two plugins that allow the connection of brokers:
• The shovel plugin, which connects queues in one broker to exchanges in another broker
• The federation plugin, which connects queues to queues or exchanges to exchanges across brokers
Both plugins ensure a reliable delivery of messages across brokers; if messages can't be routed to the target broker, they'll remain safely accumulated. Neither require brokers to be clustered, which simplifies setup and management (RabbitMQ and Erlang versions can mismatch). The federation plugin needs to be installed on all RabbitMQ nodes that will engage in the topology.
Keep in mind...
- RabbitMQ relies on Erlang's clustering feature, which allows several Erlang nodes to communicate with each other locally or over the network.
- The Erlang cluster requires a so-called security cookie as a means of cross-node authentication.
- If RabbitMQ instances are firewalled from each other, its needed to open specific ports on top of the one used by AMQP (5672); otherwise, the cluster will not work.
- Make sure the same version of Erlang is used by all the RabbitMQ nodes that engage in a cluster; otherwise, the join_cluster command will fail with an OTP version mismatch error.
- Similarly, the same major/minor version of RabbitMQ should be used across nodes, but patch versions can differ; this means that versions 3.2.1 and 3.2.0 can be used in the same cluster, but not 3.2.1 and 3.1.0.
- In a federation, only the node where messages converge needs to be manually configured; its upstream nodes get automatically configured for the topology. Conversely with shovels, each source node needs to be manually configured to send to a destination node, which itself is unaware of the fact that it's engaged in a particular topology.
Thursday, December 24, 2015
WS-Policy
WS-Policy is a specification that allows web services to use XML to advertise their policies (on security, quality of service, etc.) and for web service consumers to specify their policy requirements. WS-Policy is a W3C recommendation as of September 2007. WS-Policy represents a set of specifications that describe the capabilities and constraints of the security (and other business) policies on intermediaries and end points (for example, required security tokens, supported encryption algorithms, and privacy rules) and how to associate policies with services and end points.
- To integrate software systems with web services
- Need a way to express web services characteristics
- Without this standard, developers need docs
- Provides a flexible and extensible grammar for expressing the capabilities, requirements, and general characteristics of Web Service entities
- Defines a model to express these properties as policies
- Provide the mechanisms needed to enable Web Services applications to specify policies
WS-Policy specifies:
- An XML-based structure called a policy expression containing policy information
- Grammar elements to indicate how the contained policy assertions apply
Terminology:
- Policy: refers to the set of information being expressed as policy assertions
- Policy Assertion: represents an individual preference, requirement, capability, etc.
- Policy Expression: set of one or more policy assertions
- Policy Subject: an entity to which a policy expression can be bound
Policy Namespaces:
- WS-Policy schema defines all constructs that can used in a policy expression
| Prefix | Description | Namespace |
|---|---|---|
| wsp | WS-Policy, WS-PolicyAssertions, and WS_PolicyAttachment | http://schemas.xmlsoap.org/ws/2002/12/policy |
| wsse | WS-SecurityPolicy | http://schemas.xmlsoap.org/ws/2002/12/secext |
| wsu | WS utility schema | http://schemas.xmlsoap.org/ws/2002/07/utility |
| msp | WSE 2.0 policy schema | http://schemas.microsoft.com/wse/2003/06/policy |
Monday, November 30, 2015
MongoDB Fundamentals
MongoDB is a document-oriented DBMS. It uses JSON-like objects comprising the data model, rather than RDBMS tables. Significantly, MongoDB supports neither joins nor transactions. However, it features secondary indexes, an expressive query language, atomic writes on a per-document level, and fully-consistent reads. Operationally, MongoDB features master-slave replication with automated failover and built-in horizontal scaling via automated range-based partitioning.
Instead of tables, a MongoDB database stores its data in collections, which are the rough equivalent of RDBMS tables. A collection holds one or more documents, which corresponds to a record or a row in a relational database table, and each document has one or more fields, which corresponds to a column in a relational database table.
Instead of tables, a MongoDB database stores its data in collections, which are the rough equivalent of RDBMS tables. A collection holds one or more documents, which corresponds to a record or a row in a relational database table, and each document has one or more fields, which corresponds to a column in a relational database table.
MongoDB Storage
A storage engine is the part of a database that is responsible for managing how data is stored on disk. Many databases support multiple storage engines, where different engines perform better for specific workloads. For example, one storage engine might offer better performance for read-heavy workloads, and another might support a higher-throughput for write operations. MMAPv1 is the default storage engine in 3.0. With multiple storage engines, you can decide which storage engine is best for your application.
A storage engine is the part of a database that is responsible for managing how data is stored on disk. Many databases support multiple storage engines, where different engines perform better for specific workloads. For example, one storage engine might offer better performance for read-heavy workloads, and another might support a higher-throughput for write operations. MMAPv1 is the default storage engine in 3.0. With multiple storage engines, you can decide which storage engine is best for your application.
MMAPv1 Storage Engine
A memory-mapped file is a file with data that the operating system places in memory by way of the mmap() system call. mmap() thus maps the file to a region of virtual memory. Memory-mapped files are the critical piece of the MMAPv1 storage engine in MongoDB. By using memory mapped files, MongoDB can treat the contents of its data files as if they were in memory. This provides MongoDB with an extremely fast and simple method for accessing and manipulating data.
MongoDB uses memory mapped files for managing and interacting with all data. Memory mapping assigns files to a block of virtual memory with a direct byte-for-byte correlation. MongoDB memory maps data files to memory as it accesses documents. Unaccessed data is not mapped to memory. Once mapped, the relationship between file and memory allows MongoDB to interact with the data in the file as if it were memory.
A memory-mapped file is a file with data that the operating system places in memory by way of the mmap() system call. mmap() thus maps the file to a region of virtual memory. Memory-mapped files are the critical piece of the MMAPv1 storage engine in MongoDB. By using memory mapped files, MongoDB can treat the contents of its data files as if they were in memory. This provides MongoDB with an extremely fast and simple method for accessing and manipulating data.
MongoDB uses memory mapped files for managing and interacting with all data. Memory mapping assigns files to a block of virtual memory with a direct byte-for-byte correlation. MongoDB memory maps data files to memory as it accesses documents. Unaccessed data is not mapped to memory. Once mapped, the relationship between file and memory allows MongoDB to interact with the data in the file as if it were memory.
Database Properties
Atomicity and Transactions
In MongoDB, a write operation is atomic on the level of a single document, even if the operation modifies multiple embedded documents within a single document. When a single write operation modifies multiple documents, the modification of each document is atomic, but the operation as a whole is not atomic and other operations may interleave. However, we can isolate a single write operation that affects multiple documents using the $isolated operator.
Get more information at Concurrency.
In MongoDB, a write operation is atomic on the level of a single document, even if the operation modifies multiple embedded documents within a single document. When a single write operation modifies multiple documents, the modification of each document is atomic, but the operation as a whole is not atomic and other operations may interleave. However, we can isolate a single write operation that affects multiple documents using the $isolated operator.
Get more information at Concurrency.
Consistency
MongoDB is consistent by default: reads and writes are issued to the primary member of a replica set. Applications can optionally read from secondary replicas, where data is eventually consistent by default. Reads from secondaries can be useful in scenarios where it is acceptable for data to be slightly out of date, such as some reporting applications. Applications can also read from the closest copy of the data (as measured by ping distance) when latency is more important than consistency.
Get more information at Consistency.
MongoDB is consistent by default: reads and writes are issued to the primary member of a replica set. Applications can optionally read from secondary replicas, where data is eventually consistent by default. Reads from secondaries can be useful in scenarios where it is acceptable for data to be slightly out of date, such as some reporting applications. Applications can also read from the closest copy of the data (as measured by ping distance) when latency is more important than consistency.
Get more information at Consistency.
SOA : Service-Oriented Architecture
What is Service-Oriented Architecture?
Service-Oriented Architecture (SOA) is an architectural style. Applications built using an SOA style deliver functionality as services that can be used or reused when building applications or integrating within the enterprise or trading partners. It is a method of design, deployment, and management of both applications and the software infrastructure where:
Service-Oriented Architecture (SOA) is an architectural style. Applications built using an SOA style deliver functionality as services that can be used or reused when building applications or integrating within the enterprise or trading partners. It is a method of design, deployment, and management of both applications and the software infrastructure where:
- All software is organized into business services that are network accessible and executable.
- Service interfaces are based on public standards for interoperability.
Key Characteristics of SOA
- Uses open standards to integrate software assets as services
- Standardizes interactions of services
- Services become building blocks that form business flows
- Services can be reused by other applications
- Quality of service, security and performance are specified
- Software infrastructure is responsible for managing
- Services are catalogued and discoverable
- Data are catalogued and discoverable
- Protocols use only industry standards
What is an Enterprise Service Bus (ESB)?
- An enterprise service bus is an infrastructure used for building compound applications
- The enterprise service bus is the glue that holds the compound application together
- The enterprise service bus is an emerging style for integrating enterprise applications in an implementation-independent fashion
- An enterprise service bus can be thought of as an abstraction layer on top of an Enterprise Messaging System
Key Characteristics of an ESB
- Streamlines development
- Supports multiple binding strategies
- Performs data transformation
- Intelligent routing
- Real time monitoring
- Exception handling
- Service security
Tuesday, October 20, 2015
Real-time Transport Protocol
The Real-time Transport Protocol (RTP) is a network protocol for delivering audio and video over IP networks. RTP is used extensively in communication and entertainment systems that involve streaming media, such as telephony, video teleconference applications, television services and web-based push-to-talk features.
RTP is used in conjunction with the RTP Control Protocol (RTCP). While RTP carries the media streams (e.g., audio and video), RTCP is used to monitor transmission statistics and quality of service (QoS) and aids synchronization of multiple streams. RTP is one of the technical foundations of Voice over IP and in this context is often used in conjunction with a signaling protocol such as the Session Initiation Protocol (SIP) which establishes connections across the network.
RTP was developed by the Audio-Video Transport Working Group of the Internet Engineering Task Force (IETF) and first published in 1996 as RFC 1889, superseded by RFC 3550 in 2003.
RTP combines its data transport with a control protocol (RTCP), which makes it possible to monitor data delivery for large multicast networks. Typically, RTP runs on top of the UDP protocol, although the specification is general enough to support other transport protocols. An RTP session is established for each multimedia stream. A session consists of an IP address with a pair of ports for RTP and RTCP. For example, audio and video streams use separate RTP sessions, enabling a receiver to deselect a particular stream.
Subscribe to:
Posts (Atom)
