首页 新闻 论坛 群组 Blog 文档 下载 读书 Tag 网摘 搜索 .NET Java 游戏 视频 人才 外包 数据库 第二书店 程序员

zdg/ 


共64个网摘 [ 1  2  3 ]  下一页  |  访问zdg的个人空间

批量架构的通用原则和指导方针

zdg收录,使用标签:Database, Architecture,时间:2008-10-6 16:46:29 | 相关网摘我也收藏

评审应用I/O(分析SQL语句),确保避免不必要的物理I/O。特别要检查下面这四种常见错误:
a)每次事务都去读取数据,而这些数据其实可以只读取一次,然后缓存在工作内存中
b)在一个事务里重复读取数据,而在这个事务中早先已经读取过这些数据
c)引起不必要的表或索引扫描
d)在SQL语句的WHERE子句里没有指定键值


dbcached──“分布式 key-value 数据库内存缓存系统”发布

zdg收录,使用标签:memcached, DataBase, 分布式, Cache,时间:2008-2-22 19:02:42 | 相关网摘我也收藏

dbcached 1.0 beta* 在 Memcached 1.2.4 的基础上编写而成,也是我的第一个开源C项目。编写 dbcached 的目的是为了最大限度的发挥 Memcached 内存缓存的优势,便捷地维护 Memcached 服务器节点哈希列表,智能地支持 Memcached 故障转移,同时保证数据的持久化存储。


数据库设计方案经典推荐

zdg收录,使用标签:DataBase, Performance, SQLServer,时间:2008-1-21 19:21:04 | 相关网摘我也收藏

大型系统的生产环境,一般情况下,我们评价一条查询是否有效率,更多的是关注逻辑IO(至于为什么,回头补一篇)。我们常说,“要建彪悍的索引”、“要写高效的SQL”,其实最终目的就是在相同结果集情况下,尽可能减少逻辑IO。


Amazon发布SimpleDB:初露端倪的数据库

zdg收录,使用标签:Amazon, SimpleDB, DataBase,时间:2007-12-16 14:47:40 | 相关网摘我也收藏

和Amazon S3允许你读写删除文件不一样,通过SimpleDB你可以执行下面的命令:

#CREATE,创建一个新域来放你自己的结构化数据。

#GET,PUT,DELETE你域里的数据。还可以带属性-值对。SimpleDB在你把数据放到域里时自动索引以便能快速被取到,你也不需要预定义模式或者在加入数据时修改模式。每条语句可以带最多256个属性。每个属性值可以是1-1024字节。

#QUERY,用一组运算符查询你的数据:=, !=, <, > <=, >=, STARTS-WITH, AND, OR, NOT, INTERSECTION AND UNION。查询执行时间目前限定在五秒。Amazon SimpleDB为实时应用而设计并为此作优化。

用户只为你用的服务付费。价格方式和Amazon的Simple Storage Service相似,但是有两个主要不同:

-结构化数据存储的费用有数量级的不同。在S3上$0.15每GB月在SimpleDB上变成了$1.50。

-还有一个$0.14每SimpleDB小时的机器使用费。


说说大型高并发高负载网站的系统架构

zdg收录,使用标签:DataBase, Architecture, WEB开发,时间:2006-12-18 10:23:52 | 相关网摘我也收藏

上面提供的几个解决思路在一定程度上也意味着更大的投入,并且这样的解决思路具备瓶颈,没有很好的扩展性,下面我从低成本、高性能和高扩张性的角度来说说我的一些经验。

1、HTML静态化
2、图片服务器分离
3、数据库集群和库表散列
4、缓存
5、镜像
6、负载均衡
一个典型的使用负载均衡的策略就是,在软件或者硬件四层交换的基础上搭建squid集群,这种思路在很多大型网站包括搜索引擎上被采用,这样的架构低成本、高性能还有很强的扩张性,随时往架构里面增减节点都非常容易。


关于世界上的超大数据库

zdg收录,使用标签:DataBase, Oracle,时间:2006-11-27 17:05:13 | 相关网摘我也收藏

如果不区分操作系统环境,Yahoo! 力拔数据仓库一项的头筹,单个数据库数据大小接近 100T 。采用的是 Oracle 数据库,部署在 Unix 上, 存储是 EMC 的设备。这是 2005年的数据,雅虎现在每日接近 40 亿 PV,这个数据仓库现在应该远超 100T 了吧。 电信巨头 AT&T 的数据仓库屈居亚军。Amazon 的两个数据仓库也不小,数据量多达 24773 GB,是用 Oracle RAC 实现的,部署在 Linux 操作系统上。


构建安全的数据访问

zdg收录,使用标签:DataBase,时间:2005-1-11 17:45:35 | 相关网摘我也收藏

数据库是应用程序级攻击的主要目标。攻击者使用应用程序级攻击,以利用您的数据访问代码中的缺陷,获取未授权的数据库访问权限。如果所有其他受攻击区域都关闭了,则应用程序的前门 — 端口 80,将成为攻击者窃取、操纵和破坏数据的路径选择。本单元说明了如何构建安全的数据访问代码,并避免常见的缺陷和失误。本单元提出了一系列对策和防范技术,可以用在数据访问代码中减轻与数据访问有关的一些最大威胁。


下一代数据访问:使概念级别成为现实

zdg收录,使用标签:.NET开发, DataBase, SQLServer,时间:2006-7-27 18:00:03 | 相关网摘我也收藏

通过将抽象级别从逻辑(关系)级别提高到概念(实体)级别来消除应用程序和数据服务(例如,作为 SQL Server 产品一部分提供的报告、分析和复制服务)两方面的阻抗失谐。


了解一下 Technorati 的后台数据库架构

zdg收录,使用标签:Technorati, DataBase,时间:2006-6-30 2:56:12 | 相关网摘我也收藏

目前处理着大约 10Tb 核心数据, 分布在大约 20 台机器上.通过复制, 多增加了 100Tb 数据, 分布在 200 台机器上. 每天增长的数据 1TB. 通过 SOA 的运用, 物理与逻辑的访问相隔离, 似乎消除了数据库的瓶颈. 值得一提的是, 该扩展过程始终是利用普通的硬件与开源软件来完成的. 毕竟 , Web 2.0 站点都不是烧钱的主. 从数据量来看,这绝对是一个相对比较大的 Web 2.0 应用.Tag 是 Technorati 最为重要的数据元素. 爆炸性的 Tag 增长给 Technorati 带来了不小的挑战.2005 年 1 月的时候, 只有两台数据库服务器, 一主一从. 到了 06 年一月份, 已经是一主一从, 6 台 MyISAM 从数据库用来对付查询, 3 台 MyISAM 用作异步计算.


mixi.jp:使用开源软件搭建的可扩展SNS网站

zdg收录,使用标签:sns, OpenSource, DataBase,时间:2006-6-30 2:55:56 | 相关网摘我也收藏

Mixi采用开源软件作为架构的基础:Linux 2.6,Apache 2.0,MySQL,Perl 5.8,memcached,Squid等等。到目前为止已经有100多台MySQL数据库服务器,并且在以每月10多台的速度增长。Mixi的数据库连接方式采用的是每次查询都进行连接,而不是持久连接。数据库大多数是以InnoDB方式运行。Mixi解决扩展问题主要依赖于对数据库的切分。


Database War Stories #9 (finis): Brian Aker of MySQL Responds

zdg收录,使用标签:web2.0, DataBase,时间:2006-6-21 1:47:24 | 相关网摘我也收藏

I didn't hear that flat files don't scale. What I heard is that some very big sites are saying that traditional databases don't scale, and that the evolution isn't from flat files to SQL databases, but from flat files to sophisticated custom file systems. Brian acknowledges that SQL vendors haven't solved the problem, but doesn't seem to think that anyone else has either.


Database War Stories #8: Findory and Amazon

zdg收录,使用标签:web2.0, DataBase,时间:2006-6-21 1:44:41 | 相关网摘我也收藏

Our read-only databases are flat files -- Berkeley DB to be specific -- and are replicated out using our own replication management tools to our webservers. This strategy gives us extremely fast access from the local filesystem. We make thousands of random accesses to this read-only data on each page serve; Berkeley DB offers the performance necessary to be able to still serve our personalized pages rapidly under this load.


Database War Stories #7: Google File System and BigTable

zdg收录,使用标签:web2.0, DataBase,时间:2006-6-21 1:42:31 | 相关网摘我也收藏

Greg Linden of Findory wrote: "I've been enjoying your series on O'Reilly Radar about database war stories at popular startups. I was thinking that it would be fantastic if you could get Jeff Dean or Adam Bosworth at Google to chat a little bit about their database issues. As you probably know, Jeff Dean was involved designing BigTable and the Google File System. Adam Bosworth wrote a much discussed post about the need for better, large scale, distributed databases."


Database War Stories #6: O'Reilly Research

zdg收录,使用标签:web2.0, DataBase,时间:2006-6-21 1:36:17 | 相关网摘我也收藏

In building our Research data mart, which includes data on book sales trends, job postings), blog postings, and other data sources, Roger Magoulas has had to deal with a lot of very messy textual data, transforming it into something with enough structure to put it into a database. In this entry, he describes some of the problems, solutions, and the skills that are needed for dealing with unstructured data.


Database War Stories #5: craigslist

zdg收录,使用标签:web2.0, DataBase,时间:2006-6-21 1:34:13 | 相关网摘我也收藏

Do Not expect FullText indexing to work on a very large table. It's just not fast enough for what user expect on the web and an updating rows will make bad things happen. We want forward facing queries to be measured in a few 100ths of a second.


Database War Stories #4: NASA World Wind

zdg收录,使用标签:web2.0, DataBase,时间:2006-6-21 1:32:40 | 相关网摘我也收藏

Patrick Hogan of NASA World Wind, an open source program that does many of the same things as Google Earth, uses both flat files and SQL databases in his application. Flat files are used for quick response on the client side, while on the server side, SQL databases store both imagery (and soon to come, vector files.) However, he admits that "using file stores, especially when a large number of files are present (millions) has proven to be fairly inconsistent across multiple OS and hardware platforms."


Database War Stories #3: Flickr

zdg收录,使用标签:web2.0, DataBase,时间:2006-6-21 1:30:59 | 相关网摘我也收藏

I also asked for any information on the scale of data Flickr manages and its growth rates. Cal answered:total stored unique data : 935 GBtotal stored duplicated data : ~3TB


Database War Stories #2: bloglines and memeorandum

zdg收录,使用标签:web2.0, DataBase,时间:2006-6-21 1:29:25 | 相关网摘我也收藏

Bloglines has several data stores, only a couple of which are managed by "traditional" database tools (which in our case is Sleepycat). User information, including email address, password, and subscription data, is stored in one database. Feed information, including the name of the feed, description of the feed, and the various URLs associated with feed, are stored in another database. The vast majority of data within Bloglines however, the 1.4 billion blog posts we've archived since we went on-line, are stored in a data storage system that we wrote ourselves. This system is based on flat files that are replicated across multiple machines, somewhat like the system outlined in the Google File System paper,but much more specific to just our application. To round things out, we make extensive use of memcached to try to keep as much data in memory as possible to keep performance as snappy as possible.


Web 2.0 and Databases Part 1: Second Life

zdg收录,使用标签:web2.0, DataBase,时间:2006-6-21 1:26:08 | 相关网摘我也收藏

In this first installment, a few thoughts from Cory Ondrejka and Ian Wilkes of Linden Labs, creators of Second Life


SQL Server 2005 – Automating Creation of Database Snapshots

zdg收录,使用标签:DataBase, SQLServer,时间:2006-6-12 10:14:15 | 相关网摘我也收藏

SQL Server 2005 is packed with many new features. One of the new features that I would like to discuss in this article is Database Snapshots, which are read only static views of a database. SQL Server 2005 allows you to create multiple snapshots on a database. In this article, I would like to demonstrate the creation of database snapshots and automating the creation of database snapshots.


MSSQL Server 2005 Replication Step by Step

zdg收录,使用标签:DataBase, SQLServer,时间:2006-6-10 3:07:36 | 相关网摘我也收藏

Replication is designed to increase data availability by distributing the data across multiple database servers. Availability is increased by allowing applications to scale out the SQL Server read workload across databases. SQL Server 2005 offers enhanced replication using a new peer-to-peer model that provides a new topology in which databases can be synchronized transactionally with any identical peer database.


LINQ May CTP supports Datasets, DLINQ Desginer

zdg收录,使用标签:.NET开发, LINQ, DataBase,时间:2006-5-16 23:58:06 | 相关网摘我也收藏

The new LINQ supports LINQ over Datasets. Also, in this rev, VB and C# language versions of LINQ are available in a single download. Other major additions in this release are the DLINQ Designer, Data Binding, better support for stored procedures and ASP.NET LINQ project templates.


Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

zdg收录,使用标签:.NET开发, DataBase,时间:2005-11-15 17:20:44 | 相关网摘我也收藏

你的设置并没有问题,是你混淆了SqlCommand.CommandTimeout 和 SqlConnection.ConnectionTimeout 这两个的区别了。你的连接字符串中的超时只是设置的SqlConnection.ConnectionTimeout 的值,而不是设置的 SqlCommand.CommandTimeout 的值。SqlHelper 中并没有 SqlCommand.CommandTimeout 的相关设置。需要你自己设置。


Simple JavaScript Database

zdg收录,使用标签:javascript, DataBase,时间:2005-10-31 22:29:41 | 相关网摘我也收藏

This example illustrates one way of creating a completely JavaScript database with AMASS storage.This was thrown together and just more or less illustrates the possibility of using AMASS storage and TrimPath query javascript in conjunction for an offline in browser database. Why? The possibility of an offline database would be one of the pieces needed to create an offline AJAX application, something very desirable.


再谈树形数据的RDBMS的存储

zdg收录,使用标签:DataBase,时间:2005-9-9 1:38:05 | 相关网摘我也收藏

总得来说,NSM还是个非常不错的设计.用这种树形结构来存储怪癖图本来是一个误导,只是注意到blogger 们在怪癖点名时候有意识尽量避免重复提起已经被点名的blogger,同时大家也没有说我跟某某是一个电 话的交情,我跟某某是得花2W块的交情,拿这个模型勉强可以凑合一下,真正的社会网络用这种这种方法, 连Dijkstra都会笑话的,扯远了,打住.


奇蠢无比的数据库设计一例

zdg收录,使用标签:DataBase,时间:2005-9-9 1:37:23 | 相关网摘我也收藏

在我们添加了这两个字段之后,可以很方便的进行各种查询,添加,删除,修改,统计等等操作。再回过头来看这所谓的嵌套集合模型,真是令人吃惊,每次添加一个新的节点,都要同时修改N多个节点的数据,如果运气不好,甚至可能需要修改几乎所有的记录,删除也是一样,而当需要进行各种查询统计操作的时候,依然令人感觉到复杂与效率底下。如果每一个操作,都需要锁定整张表的时候,这样的数据库设计,就是彻底失败的!


玩得都那么有技术含量:用数据库保存"怪癖图"

zdg收录,使用标签:DataBase, 怪癖,时间:2005-9-1 22:00:25 | 相关网摘我也收藏

本来想做一个应用,专门做这个怪癖链接提交,扩散图查询的,时间关系来不及,有空再补上。 对中间交叉部分(中间联系人)的解决办法的描述。 对六度分离的数据结构的存储方式引申。 收集大量的怪癖链接地址。


Database Programming and Design

zdg收录,使用标签:DataBase,时间:2005-8-28 1:10:14 | 相关网摘我也收藏

In this article, I'll summarize 25 of the most useful SQL tuning tips for making SQL statements run faster. Although some of these techniques have been previously described in Oracle manuals and various journals, many others have never been published and are appearing here for the first time.


Get better SQL Server performance before you improve the queries

zdg收录,使用标签:DataBase,时间:2005-6-3 18:16:58 | 相关网摘我也收藏

When looking at SQL performance there's an immediate desire to jump into the queries that the SQL Server is being forced to process and see what can be done to improve them. Unfortunately, it's not always that simple. Sometimes the queries are coming from packaged software that cannot be changed. Other times the queries come from report writing tools and there's just no way to make the queries any better.



共64个网摘 [ 1  2  3 ]  下一页

zdg/相关标签



网站简介广告服务网站地图帮助联系方式诚聘英才English 问题报告
北京创新乐知广告有限公司 版权所有 京 ICP 证 070598 号
Copyright © 2000-2008, CSDN.NET, All Rights Reserved